---
title: "FDA-Grade AI - Total Product Lifecycle Control with a PCCP"
type: webinar-transcript
publisher: Ketryx
source: "https://fast.wistia.net/embed/iframe/8pp38k4z3o"
content: auto-caption transcript, proper-noun corrected
---

# FDA-Grade AI - Total Product Lifecycle Control with a PCCP

*Ketryx webinar — transcript of the recorded session.*

[▶ Watch the recording](https://fast.wistia.net/embed/iframe/8pp38k4z3o)

---

everybody. My name is Erez Kaminski. I'm the founder and CEO of Ketryx. I used to lead AI and machine learning for Amgen's medical device division, and I'm kind of the host for the webinar today. The webinar will be both kind of, I'd say, explanation discussion.

We always love to get questions as well as kind of a fireside chat with, doctor Yaron Haddad, who kind of, I think, needs no introduction. But, Yaron, I'll let you maybe introduce yourself. I'll just say that Yaron and I, are big, math math and physics enthusiasts. We met through the mathematical computing industry. Funny enough, almost ten years ago, I think I sent you an email.

Yeah. And we kinda worked together for a little bit when I was working mathematical computing. Stayed in touch. I know that you, eventually stole your company to Medtronic, became the kind of VP of AI and data science for Medtronic. But I'll let you kind of introduce yourself a little bit, and then, would love to hear a little bit about what you're doing these days too.

Thank you. Awesome, Erez. I appreciate it, and, it's awesome, being here this morning. Yeah. So let me share a little bit about my own background.

I'm a physicist by background. I did my, PhD on general relativity on, black holes and gravitational waves. And I like to say I came back to Earth, immediately after my PhD to, start a company called Neutrino. In Nutrino, we developed a way to quantify and predict how food affects people from a health perspective, by leveraging both different, medical devices and different algorithms. AI was just a part of them.

Neutrino got acquired by Medtronic, roughly six years ago. And after the acquisition, I led, AI and data strategy for, Medtronic, for three years. Among other things was the some of the algorithm work for the artificial pancreas project, which was the first class three medical device to be using AI. We've got a breakthrough designation from the FDA for this. And, I was involved in the last, few years in, cofounding, four different companies, three of which I'm on the board of and one that I'm managing myself, which is Beehive.

I guess we can, continue dive dive in further into them, if needed, throughout the conversation. Yeah. And I think Beehive, there's a point where we talk about module modularity of software, and Beehive is really a way to, build modular software with a combination of of people and AI agents in a way that makes it very cost effective and very high quality. I love the way you set it up. It's just so, sophisticated.

Right? A lot of folks are thinking that AI is gonna just solve all their problems, and, we need a little different approach to create high quality software. I'm sure you know the statistic that there's been studies that the increase in, defect rates for AI built code is, improving. That is actually something I, I I also learned from you. I think, I think it's clear that at this stage, AI is gonna play a pivotal point also in software development, but we are we are, in many ways doubling down on the fact that, AI together with humans, if you create the right processes, allows one to build things faster, cheaper, but also very high quality, medical device grid.

I always when I talk about that, I talk about the study that showed that, you know, I think only a few years ago, right, the best computer Deep Blue could beat Garry Kasparov. But then they showed that basically even kind of an average chess player with a computer can beat both the computer and people like Garry Kasparov. Like that I think people miss that because they think it's, like, black and white. It's gonna be people or computer, and it's not gonna be people or computer. It's gonna be people using computers in very, very sophisticated ways that make everything better.

And just, I think Stephen Wolfram says this very well that, computers are just a way to automate things we wanna do, but the goals and the things we wanna do are are, generally human, because of our kind of genetic history, social history, psychological history is what drives us to make a decision at every point. But let's get started. We've got a lot of content to go through. So before we start, we're gonna record this webinar. You're gonna get the slides afterwards.

We love questions. Please feel free to drop questions in the q and a area. We'll answer them as we need, live. Or at the end, we also have folks in the background sitting answering these questions. And and today, we're actually joined by the wonderful Jen Dixon, who leads a lot of our quality and regulatory work and, serve some of the largest corporations in the world trying to help them build advanced AI systems.

And then we'd love to hear your feedback, at the end about how this went and the content you wanna see next year from us. We've had a great year of webinars, so thanks to you all for for attending. I wanna start by saying why are we here? We're here because, software and automate automation software and then now AI is revolutionizing, revolutionizing industries. In particular, I think the health care industry is one that needs this type of revolution, and we're seeing that there are amazing applications of both classical machine learning and more modern large generative models that could be applied basically everywhere.

One of my favorite, implementations of AI in medicine, which I'm very, very fortunate, now to to help build or or our company helps, Heartflow build. What they're doing is Heartflow, which is a way to replace basically a physical diagnostic, with, an AI analysis of a generic CT scan. So what happens there are, patients that have certain kind of plaque buildup and other kind of issues, most around flow rates of their heart. Traditionally, you would need to go to an interventional radiologist or now cardiologists do this all the time where they basically put a lead, kind of down, your artery into the heart to check the flow rates. This is a very, you know, interventional procedure to say the least.

It takes a long time to schedule. It's backlogged. And then during this time, you are sitting there thinking, when am I gonna have some event, that could kill me? Instead of that with Heartflow, which is used by quarter of million patients every year, class two AI driven medical device, you can go same day, get a CT scan of your heart, and the same day get an equivalent result that could give you the diagnosis. And they've shown that over kind of two years, it reduces mortality rate by fifteen percent.

Fifteen percent could be hundreds of people, thousands of people, tens of thousands of families, and relatives, if not more than that, that have another opportunity at life. And it's just an amazing piece of technology, and that's why we're here. We're trying to figure out how can we make providers more efficient, factories more efficient, and also do much more in the home. For some of us who are at AviMed this year, there was a big talk about about two hundred and fifty billion dollars of health care services is gonna move to the home in the next six years by twenty thirty. And I think a lot of that is gonna be based in AI.

And the title of the talk is really total product life cycle management in AI. So I wanted to start by talking about a piece, from FDA's kind of paper about AI life cycle management and total product life cycle approach. I think that one of the big challenges of building AI systems, and your own, I have exactly a perfect question for you about it, is the fact that it's really hard to build. It's hard to build if you're building it for for, you know, social apps. Takes a lot of power, a lot of energy, a lot of planning.

To build it in such safety critical environments, both for classical machine learning and for generative, AI models, requires kind of a really, change in the way you work and the speed in which you work, the scale of the deployment, and also how you monitor it. And with generative AI models, it's not clear if you can validate them in the classical sense of the world, which is to, you know, kind of prove again and again they provide the same result. I think a lot of the work with generative AI will be focused on risk management. If I was part of the committee that wrote TIR three four nine seven one, risk management for, medical device AI systems, and I think that was a big realization there is, you know, we're gonna need a lot of risk management, and I'm happy to see the FDA coming out with a paper saying, what we really need is risk management and a total product life cycle approach to things. We'll dive deeper into what TPLC really means, but for the FDA from this paper they released, in October, they're really talking about planning and design, data collection, model building, V and V, deployment, operation, and then kind of, real world performance evaluation, post market surveillance, the classical things people do, but now we'll need to do them faster and better with higher fidelity.

And that leads me kind of to the first question I wanna ask you, Roan, which is, if AI is so impactful, and we know today from that Heartflow case study and many other avenues of our life, it is very impactful, why isn't every company incorporating it into their products? What are the primary challenges you faced integrating AI into medical devices? Yeah. So that's a great question. I think, at this stage in the hype cycle, I mean, everybody's, talking about AI.

I think most companies want to incorporate AI or in the process of incorporating AI. However, we do see that in certain industries, the penetration of AI is significantly faster than others. And I think part of it is for the natural reason that, once, specifically, if you're talking about med tech, once you want to integrate things, like AI that are statistical in nature into processes that involve human life. It's it's pretty nontrivial to to build AI at scale, you know, medical grade, that works in a regular environment and maintains safety fundamentally because of that statistical nature of of AI, I think more more than anything. Now not to mention that, typically, companies are trying to leverage the same cycles for development that they're used to and the same, processes that they're used to to develop AI.

And I think there is a lot of, value in, approaching it differently. I think that's one of the things that attracted me to to what you're doing, because fundamentally, part of the ability to, develop AI, at quality and faster very much is is very much, related to, the process that one takes in order to build the systems and the ability to actually monitor it and, prove its, its, you know, its performance over time. I think Yeah. And validated with intended use continuously and doesn't have too much drift. It's interesting.

When I came to the med tech industry from tech, you know, I I was part of, you know, played a small role, but part of role of building systems like WolframAlpha are supporting their continuous development. These are massive infrastructure pieces. Right? And Wolfram Alpha has developed almost over twenty years now. It's quite a beast.

And when I showed up into the med tech industry and the life science industry, I said, wow. Like, they build software that is so much smaller than what people are building in tech, maybe a thousand times smaller, at least, hundred times smaller, kind of that kind of scale, and they're releasing it a hundred to a thousand times slower. Like, what would it look like if they scale up and have a need for these massive infrastructure pieces, these massive applications, that are coming into med tech and life science right now? And I thought that would be really, really challenging to do given the way people working, which is why I set out to build, this tool. And I know you made a similar move.

Right? We both kind of started our career in physics, ended up, working in in the tech industry and then moved into med tech because I think it is very exciting to work in health care. And I wanted to ask you, what are the the most striking differences you saw in the release cycles? You know, how often did you release software before and after? Yeah.

So, you know, initially, Nutreno was a start up. At the very beginning, we weren't a medical start up even. So the release cycle was extremely frequent and very fast, sometimes even, multiple releases per week. That was what we were striving to do. But when you are playing the game under that setting, you know, the the the the price of potentially breaking something is much lower.

Okay? And after we, got acquired and integrated into Medtronic, naturally, we had to, you know, support, much, much heavier, let's say, quality processes as part of this. And basically, what you do, you're trying to develop in the same way, but all of a sudden, hey. You added a bunch of different steps to a process that, in many ways, not only, I think, slowed down, engineers and what they're trying to do, but also burns them down in terms of the that's one of the things that I found to be really interesting there. But, we added a lot of steps to the process, that really slowed us down.

And I think, Medtronic was, wise enough to be like, hey. You guys you you should consider doing things, differently, obviously, to the same level of regulation, safety, documentation, etcetera. But don't use all of our legacy systems for everything. Try to do things, differently if you can. And they gave us access to amazing, quality and regulatory talent in in their company to help us build new processes, in how we do things.

And I think that was a a big part of the success, with Nutrieno with the Nutrieno integrated into Medtronic and being able to develop, much faster than the rest of the of the company. Yeah. And I think the team there just uses, you know, is very modern in in the way they think about the problem, the process of their tool chain. We're very fortunate to be working with them. There's a great case study on our website about what they've been doing with us.

And I think it is a a nontraditional product developed within a company that is reinventing itself all the time. And that scale, it's it's crazy to think we're still able to do these reinvention, which just shows you how amazing of a company Medtronic is, especially with the risk they take on. Right? Medtronic, I think, is amazing. It's just saying let's attack the most complicated, dangerous problems that maybe most other companies couldn't even get through the premarket authorization process, but we know there's a patient need there.

And and we're also willing to experiment with new things to try and do that better and faster. I think not enough companies at that scale are trying to do that. Very, very hard. And let me ask you another kind of a a more pointed question, which is, you know, when you look at this life cycle on the next slide, I'm gonna break it down even more. What do you think are the the key challenges that developers faced, when they're moving from an unregulated environment to a regulated environment.

I always call the transition from being a regular developer into a regulated developer. And I think there is a generation of devs who want to do this, who wanna get into life science. They're kind of they're burnt out at working at ad tech and fintech and many of those techs. They wanna do something really meaningful. But it is a very a very big challenge to them.

So how do you think about that from your experience? Yeah. It's a great question. I think that, one of the things that we noticed is basically, as I mentioned before, you're adding steps to a process. And typically, these are steps that, developers are not used to do.

Definitely not at that level of, robustness and thoroughness, that very much deviate in a way from what, developers are actually used to doing and wanna be doing. Right? And what so I think it's very easy to get people excited about building devices that save life and help people diagnose disease, etcetera. But once, some people make the transition from, you know, typical tech to medtech and understand the, you know, the additional hurdle, of, of development, it it sometimes becomes harder, you know, it's a little bit of an elephant in the room, but, like, to to maintain, talent, when doing that. But that's something that we definitely noticed.

I mean, part of the challenge is is you you need to add more steps. You need to to build safely. You need to, you know, document at a at a completely different level, but yet you still want to innovate. So those things almost seems like they're in conflict. And in in in in some way, this is the reason that traditionally, Metac is a few years behind some of the innovations that you see in, in other nonregulated industries.

Yeah. We we often hear a number that I think is not you know, there's no study to back it up. It is my personal experience, my team's personal experience talking to companies, hearing, HR folks at this company's talk. I'm sure that, you know, no one really wants to talk about this elephant in the room. We hear a lot of people talking about a fifty percent, kind of first year developer churn that people are just not staying in these companies.

A big part of it, I think, is just the, percentage of the time they spend documenting is very challenging for people who are not used to it. And Yes. What they see, you know, isn't there another way to do it? And I think for us, what I love about the tool we've built and the way customers use it is we have flipped that paradigm where we've reduced the vast majority of documentation time people spend, and we're seeing that acceleration, and we're trying to get some studies about the retention issue as well. Let's dive deep a little bit into this, model by the FDA.

Again, great white paper, great blogs about us by FDA. We'll do a whole series at the beginning of next year diving much deeper into different parts of this. But this is what the FDA is starting to talk about for AI life cycle and, critical quality attribute interaction. How do you measure and make sure that the process is good and have this holistic TPLC view of the world? So we start from planning and design.

There's many different aspects for planning and design, the more traditionally right intended to use problem definition. And then there is all these unique aspects for AI from feature engineering to fairness, what metrics you wanna use. That metric will come up again and again and again as part of your predetermined change control plan. That new guidance that came out just, I think last Wednesday or the Wednesday before last in final form saying, if you can show us you have this level of control and you know what to measure and you have approved that predetermined change control plan with FDA, you can make that change really regularly. Moving on, there's all these aspects of data collection and management, making sure your test set doesn't intermingle with your, training set, making sure it is fair.

I think, Yaron, and I bet you agree with this, that one of the biggest issues people are gonna face is they think they're serving a wide and diverse population, but then practically when they launch their products, they launch them in Boston, Cambridge, New York, California, Minneapolis, right, San Diego where the big hubs are. And then their data very soon becomes very, very skewed towards those populations, and I think that's something that, you know, has to be addressed early on and in plans. Uh-huh. Yeah. You then need to model, build, tune the the model, perform verification validation, deploy the model, monitor it, and have all the real world evidence from complaints to any other metric come in.

And we know now study has shown that the lack of, continuous improvement of these models and refinement is causing them to deviate and drift between hospital systems, between pharmaceutical factors. We now know that if you change a gasket, the model completely breaks and you need to retrain it if you're planning on using any models as part of your biopharmaceutical, pharmaceutical manufacturing activities. I wonder, Yaron. Like, I feel like this is stuff that is talked about in the tech industry and is done to some varying degree. Like, people who do a lot of AI, they do know all these things.

Just it's like anything between med tech and tech. No one is really, like, taking it to the to the limit of I'm about to inject this into a patient. I'm about to put it in someone's body. It's gonna govern someone's health. So, you know, we're kind of like in tech, usually, we kind of run around it and say, you know, but it's not gonna kill anyone.

So we don't need to invest so much time in that. But I don't think anything here in the FDA's list jump jumped out to me as as being kind of overly burdensome as compared to kind of making a lot of sense. Yeah. I mean, in many ways, everything that is listed here is is and has been considered best practices for AI for a long time, and I think, you know, that's what the a FDA tried to do. I I sat with them in the first, meetings and tried to advise them on what things have to be, part of that, list in the AI life cycle.

But the reality is that the majority of the company don't have the time, the resources, or the patience to go through every single step that is shown here often. So then you see companies of different sizes depending on the amount of resources and the, you know, the deadline they have for launching a specific product. Basically, remove sometimes removing steps from from the process. But but at the end, it is what's been considered the best practice. Yeah.

I think so as well. And I just wanna highlight here what does Ketryx, where does Ketryx get involved in? There's a lot of places, but, kind of the the strength in our pro product that is not really addressed by anyone else is these places around the general kind of aspects of running a life cycle. And then we can also connect to the other parts like data collection and management to execute that. And I think this is a really good drive into what is the TPLC really all about.

And what it is, it is a way to think of end to end control and oversight of products. You can call it creating, transparency. You can call it creating, end to end traceability, but it's fundamental fundamentally a way that the FDA is saying, show me that you know what you're doing, that you understand what your product does, and that you have ways to test both at the end of the life cycle and throughout the product life cycle, the quality. Because we know already that there's many tests that, you know, you can't test an airplane at the end because most components will be hidden. You have to test it as you build it as well to check that that, you know, small little gasket that's internal to the plane is correctly fitted and so on with a quality check.

And that's what the FDA is asking for. Tell us, what is the product do? How is it made? Like, what's your processes and steps, and even how does it work? Who's working on it and why?

Why are they well trained for this task? And then, how do they evaluate safety and performance? And I think the FDA is correctly pointing out that, generally, these functions are spread out across the company in many different teams, and those teams are a bit separate. And the FDA says, well, that doesn't matter to us how you structure your company and your development team. What we care about is the final product and its result to patients and your ability to improve that product over time and how fast you can run that cycle.

That's now gonna be a question for the companies while still maintaining all of these aspects. Right? The FDA is not gonna tell you, oh, because you're doing AI and you wanna release every day, you should stop doing these activities. It's gonna be the other way. You need to get better and better at this.

And I think, related to a stat article I wrote earlier this year, I think that part of the problem is that many medical device companies established the regulatory processes and quality control and quality assurance processes many decades ago, mostly in the late seventies and eighties. And now they're gonna move towards, you know, a more faster life cycle, both for their products and for their, AI systems, especially with this new predetermined change control plan that we're about to talk about. And the complexity is just going higher and higher. Right? Every year, we just get more complicated systems.

How do you typically see companies manage that that complex complexity? Yeah. So, unfortunately, companies take an approach that I'm not very, very keen on because I think that it doesn't scale very well, in this case. I'm talking specifically about the enterprises. I'm not talking necessarily about the the smaller companies.

But as you said, many of the medical device manufacturers build those processes, you know, thirty plus years ago, and they're trying to implement similar the the basically, the same processes and same approaches, for, problems that are, somewhat different than what they initially intended to to solve. Talking specifically about AI. And as a result of this, basically, the amount of overhead that is necessary just increases over time. It's cumulative. And what you see a lot of, bigger companies, doing is, you know, throwing more and more people at the problem to to assist, manage that.

And, you know, in many ways, you can do that. And to to some extent, it solve a small part of the problem, but but that's a fundamental challenge that at the end at the end, you're you're kinda how do I say this? You know, like like, you're you're dragging a lot of, additional weight with you that is unnecessary. And I think this is where companies should actually focus a lot on both automation, and we can talk about it in a moment, to facilitate a lot of these processes because there are ways of automating some of the processes that the FDA wants, to see companies doing, and also to to try and stay relatively nimble in terms of the teams so you can innovate faster. And this is, I mean, you see probably the biggest incumbent that, that has been very famous at doing this is a company like Amazon.

You have you know, Bezos had this, two pizza tray, rule where whenever they wanna build something new, it needs to be a team small enough that it can be fed by two pizza trays. You you don't often see that in med tech. You know? Usually, you see, hundreds of people or at least many dozens of people, trying to innovate. And I think this is really where automation makes a huge difference.

Automation and, you know, being willing to to also establish some new processes for certain things. Yeah. And I think I've yet to meet the the meta company that has enough software developers, software development leaders, software quality assurance, design quality assurance people, and they always wanna do more products and there's a cement pressure. And if we could just increase the automation and productivity, it's not like they're gonna have less people. They're just gonna do way more products with those people because they have such a big roadmap ahead of them.

Every large MedTech company, I think, struggles with that. And, I love a discussion I had with a a customer this year where they said, you know, we keep talking with documentation, but, like, we're talking about thousands of pages of evidence. Right? Tens of thousands in some cases. It's not like a a trivial amount of work to do just the documentation for a specific version.

And this is where I love to introduce the the way we think about TPLC. So, right, TPLC is all about control and change the product. And what it really is is one way of looking at is this diagram where you have kind of these four quadrants of product design, supply chain, manufacturing services, and operations, and post market surveillance. You have these three arrows. One is the quality management system, the set of rules that drive you across this process that you sometimes need to verify or followed correctly, the risk management, the risk management activities, the risk based processes you're executing across that life cycle, and then, of course, the individual variance, of the product, whether that's an EU product or US product, whether it's for males or females or or many, many other variants of versions and deployments and features and combination rates, especially when you get into, like, large capital equipment medical devices like, surgical robots.

There's a lot of variants going on out there. I think the variants are a very interesting point because it's one of those things that is basically by majority ignored in the tech industry. Right? Like, most big tech companies are not thinking that they have thousands of variants deployed right now, and we should analyze and understand how they're all connected and the effects of them. But in medicine, it's actually quite important.

If I have a variant that's failing in a very, risky, very dangerous way in one country, I do have the requirement and the responsibility for the public, to go investigate that and be able to modify other variants based on that because I'm already knowing it's failing somewhere. And that's not just a regulatory requirement. It's it's like a human society requirement. Like, people are dying somewhere. You you don't wanna, like, just let it go.

And I don't think anyone works in medicine over lifetimes because they just wanna let it happen. And now diving into this diagram a little bit is if we just focus on one quadrant one quadrant at a time, this product realization, we're talking about launching a project. Right? Setting up requirements, setting up specifications, how those requirements are met, performing verification and validation whether of our process or risk management or actual variant is kind of validated and verified correctly, and then running a clinical trial usually to a specific variant or a baseline. And I know, Yaron, that, this is a much more elaborate way of working than how people work in the tech industry.

Right? We don't really use in tech specifications. We don't do a full V and V. We definitely don't do a clinical trial, just not how we think about the world. And so when you were acquired, you're basically writing stories, code, and tests, and testing the aspect you think are appropriate.

Right? It's not like you write a set of requirements. That's just not how the agile software development mind works. And as this acquisition is going on, the sophistication of your product design process increases substantially. Right?

The amount of steps, the amount of of control. Can you share with us a little bit about the challenging of how the challenges of how you're aligning an existing product design process to that of an enterprise, and now you're you're being deployed at this very big scale, and how do you think about it? Yeah. Absolutely. So, you know, in many ways, as a start up as a tech start up, you, basically focus much more on, on innovating and moving as fast as you can.

And and often, not always, depends on the use case, but breaking things or, is to some extent okay as long as you, as you did in the right place and you move fast enough in order to fix things. However, once you start, once you need to align with, an enterprise, it actually means trading, some of the intuition and the speed that we always emphasized, for justification. So basically every design decision, a story, every test, everything had to be documented extremely well and well thought out of. And, before that we were we could have been looser about that in many ways, about our definitions and the requirements. So it was a very, tricky challenge for, for a young company.

I think it initially all of a sudden, we realized it can significantly significantly slow us down. I remember conversations with the team. People were concerned that it's, you know, it's gonna do like five, six x slower development just because of that. So also for in many ways for the acquirer, you know, it doesn't benefit them, as much as it could. I think, this is more or less was, in sync with the time that you and I, start talking about, Ketryx.

So and I maybe I should say it, transparently, but, Neutrino became, one of the I don't know if, one of the probably one of the first users of of Ketryx. And it came exactly at the time when we had a lot of concerns about the ability to actually innovate and move quickly while doing it, while doing it properly in a in this regulated environment. And I think, this is, where, Ketryx as a tool played, you know, a huge role in in, Neutrino's ability to become, like an excellence an excellence center in, Permitronics, and to continue innovate fast. Obviously, there were there were other benefits. One of which I mentioned before, my team was not, team that was, let's say, that had any experience.

The majority of the team didn't have experience with with quality management system. Some people did, but the majority didn't. And there was a lot of, concern around retention related to this because as we put new new steps, you find you find out how how much, developers, that are not familiar with it hate it and how much it prevents them for doing what it is that they're passionate about doing, which is like building things that are impactful. And I think, that was part of the challenge that we eventually was were able to, solve with the with the integration with Catarix that really made a world of a difference in our development process. Yeah.

And I think the most important part is just an amazing team, on the all side that is is just so forward looking, so developer first, so dedicated to the work of medicine. And and understand the responsibility because the size of the deployment is just you know? Mhmm. It's hard for people who who first get into a company and someone tells them, you know, like at Amgen. Amgen deploys to tens of millions of patients all the time.

Like, you know, you're making drugs that are used by people at their the most critical point of their life. They have cancer treatment, severe illnesses like rheumatoid arthritis. It's it's hard to understand that sometimes how scary that is that you're deploying technology or products that are used by them and the responsibility. So just moving forward in this life cycle, you get to this clinical trial just to to keep going on this TPLC. Right?

You have a supply chain. Now the supply chain, of course, comes also in the clinical trial, but also afterwards into all the different variants you're making. For software, right, you need to produce now this SBOM, a lot of regulations about SBOM and the type of audit you need to do your supply chain, configure your client, configure your CICD pipelines, make everything. One of my favorite features of Ketryx is we can integrate all these different steps into your CICD pipelines to prevent you from kind of going to product launch without doing all the steps in the right sequence, which basically prevents you from, being out of sync with your processes and having kind of quality assurance deviations. And then the next step is is post market surveillance.

So now I've deployed all these variants. At the end of the day, you don't deploy a system. You deploy a particular variant of that product line, and then you get complaints on them, of course. Large companies, this is, the business is post market surveillance, some might say. I remember, a gentleman, from your team, we talked about it.

He said, you know, if you're worried about validating the first system, you have a lot of problems because the validating of one system is the that's a small part of the work. The problem is the change management between one validated system to another between that manifold and configuration space to the next manifold and configuration space. And then, you get a lot of complaints, of course, because your product is widely used. We're talking about, in some cases, hundreds of thousands of complaints per year or per month. Then you need to concentrate all that information with your limited resources to decide about the corrective and preventive action you need to take if you need to take or the change requests you need to write for the change orders for the next product version.

That is a very, very hard step in such complicated systems and where I think, why the FDA is so focused on TPLC because they wanna see that you could do that. Right? Show me a complaint. Show me the threat of that. Show me how you do all this process, which we'll show in a demo in just a few minutes.

Now going in from this into the other part of discussion of this PCCP, predetermined change control plan, again, the final guidance just came out about a week or so ago, maybe two weeks. We've been waiting on it. Another exciting thing that happened is I think the FDA is using their regulatory authority to authorize not just changes to AI, ML, and software systems, but also in, the twenty second of August of this year, the FDA came out with a draft guidance that says, we're gonna allow you and by you, I mean, manufacturers to do these type of changes that we're describing here to all products and accept high risk components of a premarket authorized device of a PMA of, like, an implantable or very high risk device. And I think that's gonna change how people are thinking of doing medical devices. It's almost in contrary to the way it works today.

So, just explanation what a PCCP for AI is, it would be, you know, I have an approved model and then I wanna retrain it. Just for reference in most tech companies, even small small startups, this retraining happens on a daily basis. Every night there's a build going on. They take all the new data from the app. They check if the model is better or worse than the previous iteration.

If it's better according to symmetric, they publish it. That's exactly what the FDA is talking about doing here, but just with more control to make sure you're not mixing things around. So you have an approved model, you get a lot of new data. Models get better with data. This is a law of, nature I think by now.

At least it's a heuristic of nature. They just get better and better if you have this. And then you need to separate the model, check for its data quality, segregated into two different buckets, test and training. Make sure you're not mixing the test and the training. There's no duplication.

And then you wanna retrain the model. You get a new model, and then you have your metrics. We talked about that earlier. That's a core part of the PCCP is these metrics, and you wanna make sure that you're meeting that metric, that acceptance criteria. You do an impact assessment.

Everything seems right, and then you release your product. Doing this on a daily, weekly, monthly basis is a very, very hard, with the level of control documentation, part eleven compliance required here. And diving even deeper into that, just the one part of this is the data data set management part of the world. It's you know, you have this, model that is out there in the world. Right?

It makes predictions. Again, it creates kind of there's more and more data being created. You then wanna use that data, to train your model. So you split it to a test data and training data. You train your model.

You evaluate it with a test if it meets all the metrics, and then you replace it. During this replacement step, you need to follow all your V and V procedures and make sure you're testing everything. This is also true for the kind of data system. Right? So if you are using a pretty, you know, I think huge majority of companies use this tool chain, you're using, like, Jira to manage a bunch of the software and system requirements, Jira and GitHub to manage, you know, some of the model specifications and performance, the architecture.

And then you have datasets. Now the datasets can, you know, exist in, like, an s three bucket that you can use some dedicated tool. It can be in a in a in a SQL database, post graph database, and then you train the model. And now you need to go all the way up. Right?

Make sure the data you trained it makes sense, and there's no contamination between training and test. Verify the model. Check that, you know, it does all the tests you wanna pass. I remember recently the anthropic CEO, in Lex Fridman's podcast described this as a whack a mole game, where you're trying to figure it out, and he's kind of worried about people using things like this in bio and nuclear and and safety critical industries, even though that's the industry is that kind of need these things the most almost. And then, of course, you go up the way into, like, the design validation or system validation step where you're making sure that overall, the thing works.

And if you think about it from the overall approach of managing AI systems, especially for Gen AI devices, I think, you know, it's it's the same kind of thing. There's just more need for control because it's harder to validate. Right? You will have some measurements of accuracy, but the models don't necessarily, are not as, you know, recurring in the response. They won't necessarily always provide you the same thing because they're not just statistical like classical machine learning.

They're like hyper statistical. They're even more flaky than how it used to be. And then you need to think of the transparency of the in the premarket evidence. How do you measure the performance for the specific use case, the specific GenAI technology, and how do you are you planning on doing post marketing monitoring both for continuous evaluation if it works, getting feedback from users? Many medical apps today don't have a feedback button yet.

It's because they don't wanna get even more complaints, but that I think will be a big part of this story. And then how do you kind of make sure that you don't have biases by regions and by evolving models by changing foundation models you're using? And, Yaron, I wanted to ask you because I know we've had this discussion a few times. Knowing what you know about, you know, class three AI, all these different risky use cases, how do you think of approaching using a large language model in a med tech pot product and and testing all of this, making sure it works? What would you focus on?

Excellent question. Generally, okay. This is obviously, in many ways, is, is often nonprevious because if you wanna compare results of large language models on different cases, sometimes the results don't end up being identical, but they may they may both be okay. Right? Because language is more loose than, let's say, predicting a quantitative, value.

Basically, for me, what I like to track the most is, I I like the systems that we developed to have very robust monitoring, tracking of everything. So we typically like to keep all real world data, that we're using with with actual patients, okay, for, that after the fact analysis. And often, one can actually even use large language models in order to do comparisons between different results, that you get to in order to make sure that they still fall within the same buckets of, of safety that one would have. So I would say those are the two, favorite things in many in many cases, not all cases. There is no there is even opportunities to build, deterministic ways of, of, comparing the results, but not not all the time.

Yeah. And I think it's just such a big field that's gonna be opened up to to figure out how to do this and how to, you know, kind of recursively use, generative adversarial networks and all these formal methods to figure this out. Sounds good. I think it's gonna be very hard, though. So, Jen, just to kind of finalize this before we go into the demo, I think sometimes it's important to mention why is doing a PCP gonna be so hard.

One is the existing quality management system. The process are not built for this. Tools are not, built for this. And then, post market surveillance aspects is gonna be very, very difficult because of the complexity. There needs to be a lot of expertise around how do you do model validation, how do you govern these processes.

And actually doing this, which is what, why I spent the last kind of five years of my life trying to build a system to do this, requires a very complex life cycle to move very, very, very fast, which is kind of in striking contradiction to how things are developed right now, especially in software and life science products. Before we go into demo, I always you know, I don't like leaving the story with, like, challenges without any any solutions. I'll say that there's, you know, three things we recommend teams do to solve these problems. One is make your architecture ready for this, have a way to automatically generate all the evidence at a click of a button, and then integrate the risk analysis and the the procedures into the way you manage your configuration. I know you're on this first part of something that's near and dear to your heart, and it drives you crazy to see systems that are not well architected.

So maybe I'll let you kind of talk about this for a second about how you wanna see an architecture built for change. So, just describing this right. There's a patient and a provider app. There's two different product systems. Then they rely on different software and electrical systems, and they have these different kind of underlying services.

Sure. And I'll try to be quick. I know you wanted to also, show the demo, so thank you. Yeah. So I think this this is you know, everybody's talking about, modularity and decoupling a system, but I think that in practice, very often, that's where a lot of companies make, mistakes, unfortunately.

It's a highly non trivial problem sometimes to build a system in a way, in which you can, replace or improve sub subsystems without having some kind of, yeah, some kind of an effect on on other services. And this is something, that was very important for us in Neutrino. In many ways, it's a it's almost a fundamental principle in my in my current company that I can mention this in a second. But I like to basically try and perceive future requirements and features to the extent that you can, and the architecture process in such in which each one of the pieces, each one of the pieces can be tested separately, decoupled, with ability to validate it separately. And in Beehive, maybe I'll just mention in a in a couple of sentences, we, have a product.

We also do consulting about this, that, allows companies to integrate their favorite, software development tools, anything from, Slack to Jira and Trello and Monday, etcetera, Figma, and to basically transfer to our system any kind of, request or wishes that they have on things that they need to develop. But the system is built in such a way that fundamentally modularity is, is like a a key a key, principle in the system. So anything that is designed is designed in a way that pieces are fundamentally independent, and we use both AI and human in the loop in order to build those pieces. So you can test each one of them separately. And it has, like, a double value.

There are two values for this. One is you can start distributing the software development work and build it much faster than you could otherwise. And secondly, in a in a especially in a regulated environment, it it allows you to actually make those changes, faster but still safely while you can take test each piece, separately. And, yeah. So that's that's something I'm I'm I'm, doubling down on, I guess.

I think it's very important, and I I could show you that it's not just theoretical. I'll skip this slide. This type of work, the system of system microservices approach can reduce complexity significantly. And we, this year, worked with this great company, Heartflow. Again, two hundred fifty thousand patients a year.

And within the first ten weeks of the engagement, we reduced the complexity of their life cycle by the number of items they need to manage by ninety percent through the system system approach, and they can move faster and they would agree that it's safer and higher reliability. But you could read all about that. What I'm gonna do right now is I am just gonna jump into the demo because I really wanted to show folks how do you actually do this. And here is an example of Ketryx. Like you could see here, Ketryx has different kind of project set up.

We have two projects in the system, this monolithic product that I'm gonna get into as well as this system of systems here on the right side that we're not gonna jump into today. In Ketryx, you have organizations. Organizations have members and groups. Those groups allow you to manage projects and have different roles like quality assurance, system engineering, r and d, developers, architects. And through those membership of groups and proper training, you gain authority to perform actions and projects.

And what's a project? A project is kind of a set of, configuration items. And we see here a bunch of configuration items coming from Jira, some from GitHub. We can ask all kinds of questions like show me the difference between version two point one and version two. Show me what's new, what's changed, what is a risk control that's newer changed.

None. What is missing tests? You know, just two things are missing tests. All these kind of complicated questions. And then you could basically also see, the traceability of this.

So I wanna see the traceability by use cases. I have all kinds of use cases here. They trace into design inputs, design outputs, verification validation. And these are really, in this case, Jira tickets. They can be other artifacts and other systems like ADO.

And you can see here kind of the different, aspects of this life cycle, the different control we have where you can change the state. The state is a rule based engine, and we basically create traceability, Alcoa plus plus type auditability, and the ability to kind of generate evidence from all of this as well as immutability. You see here this kind of approval palette that can change over time, and then you can see here the, traceability of this, particular item, which is coming from a risk. So it is a risk control, this requirement, and it is the parent of another requirement. Now I think for the total product life cycle, right, one thing is you wanna do is look at this design, product design.

Right? Going from use cases all the way down, generating evidence as a as a word document or Excel or a CSV. Another aspect you wanna see is what if you have a complaint? So we created here a little dashboard for complaints. So I I produced a version, but then I have a complaint.

That complaint could be in Jira, could be in Salesforce, and could be in other systems of that nature. It has all pie kinds of information about what happened to the complaint and who reported this and why. And then that complaint leads into a change request. That change request has certain risks associated with it. There's then a specification that is affected by this change, and then there are tests to that.

So this is the change request that was produced based on that complaint, and then that is what we're introducing into our version. Now if we go back into the Ketryx version screen, we can see that we have certain versions. Some of them are actively being monitored. So this, for example, is a posted, a live, kind of, version that we are monitoring its SBOM to see if there's any issues we need to fix. And then we have a new version we wanna work on.

We see all the progress. We are actually ready to go. Right? Design controls are ready. Test executions are done.

We see there's only been, kind of two new items, two changed items in this version. We can automatically generate all the documentation if we want. Very easily, this could all be configured to your processes and your company and how your your company sees your work. You know, Ketryx is really a way to enforce the processes you already have in the tools you're already using, like Jira and GitHub, and automatically generate all evidence of compliance here through this documentation screen. And in this particular version, we're gonna create an incremental release.

So that's exactly that PCCP style release where we're doing an incremental release. There's really nice explanation of what that means. We're only gonna change three items in the baseline in order to minimize the amount of change. And then we also wanna automatically monitor this version, in order to make sure that it's, it's being looked at and we're tracking if it has any vulnerabilities against it. So let me just go in here.

And now that we we've done through the process, we kind of read through the complaint, created the change impact assessment, the change request, sorry, right here, created the verification test for that. Let's just approve this version with all its documents. I'm gonna sign it here, part eleven compliance signature. I'm approving it. Confetti flies in the air, and we now have a version with all its documents ready to go.

Again, here specifically, I actually disabled the approval to make it easier to demo, but you can also require people to approve it, approve it in a sequence, and so on. Last cool thing I wanna show is is how can you get better and better at writing change requests? So one thing we could do is just ask it. Hey. I know that k d one four seven, this ticket, is the complaint.

Can you, Ketryx Intelligence, create a change request for k d, k d one four seven? Let me just fix that extra r here. And then I could ask it, and it can go grab that life cycle from another system. It could also mix and match life cycles from different systems configuration items. It creates a nice change request, and then, I can approve of that.

I can also ask what items are affected by this change. It will look across my life cycle across the complete TPLC, start talking about those changes, and then I can post it to the system of work. Other great things it could do, like update all the relevant tickets, redline them, and help you understand what to do. And that's kind of it. I just wanna show a quick demo here.

This is gonna be a big part of our, demos, in the coming year. We're gonna have much deeper dives into how to do different parts of this life cycle. I also wanna leave people in the room with a solution. If you were trying to think of how to do predetermined change control plans, here's a free template that we know people are using, with the agency, people are using internally. We love to support this, so please ask us if you have questions.

And this is, of course, free available on our website like all our education and resources. And I just wanna invite you to our next webinar that will happen in January about navigating, TPLC framework for generative AI devices. So a much deeper dive dedicated for generative AI that'll open this entire kind of, multi month webinar series about how to build and deploy, AI products at scale. And if you're here in Kendall Square in Cambridge, Massachusetts, we invite you to come, to some of our, in person sessions that we're gonna run, here. So if you work for any of the companies that are located around us, in Cambridge, Massachusetts, and Boston, we welcome you and thank everybody for your time.

Finishing right on time. We'll stay here to take any questions that folks have, and I appreciate you all staying with us, in this kind of week before a lot of companies go on break to learn a little bit about deploying AI systems. And thank you so much, Yaron, for, spending time with us today and helping sharing your knowledge of how to do this. Thank you. Thank you so much for having me.

I think that, you know, companies that do it right have a huge opportunity to make a impact on more people's lives than otherwise. And, I would love to see companies implementing innovative solutions like, Ketryx, in building building building better and smarter in many ways. I have to run to my next, session folks, I'll So, folks, I'll be here to take some questions for the next two, three minutes if there's any relevant questions that the rest of the team can't answer. And I thank you all for coming and attending, and I hope, to see you in another webinar. And I welcome you, to try the product.

I see a great question here I'll answer live. Do you support the validation of Ketryx as a tool used in medical device design? So we don't just support that. We can also provide you, with a a validation package. Ketryx itself is the only IT system in the world that is not a medical device that is certified to six two three or four, and we provide medical grade medical device grade design controls, to every every claim we make against the system.

So, for for our enterprise customers, we provide all of that in a package. But, yeah, absolutely, we provide that. Thanks for the question. I mean, I'll give it another two minutes to see if there's any questions. I see a great question here about support for migration.

We do support migration. Again, we the question is how do you support migration from existing tools to Ketryx? We actually support, connection to a lot of different tools. So we don't always you don't always need to man migrate. And then, that's the idea.

Right? Ketryx is a way to use the systems you already have, the processes you already use and automatically generate evidence of compliance. So, you don't always need to migrate, but we also have services that allow you to migrate, both independently and through us, if you wanna work together. That's a great question. And then another great question from Miguel is, can the system be adapted to other countries like Japan?

Absolutely. It's already being deployed in the EU, US, Japan. And what's cool about it is we can configure your entire release so that a click of a button, you create a region specific dossier with all the information you need, which I think is just a very, very powerful way of doing it. I see questions still coming in, so I'll wait here for another moment. Okay.

Good. Well, thank you everybody for the time today. I wish you a wonderful, holiday break for those celebrating all the different holidays. Oh, and I see a bunch of questions coming up, so I'll just answer these. During development, a lot of document has been created.

How to ensure those document FDA requirement? Well, I'd say that the better way for that is through Ketryx. You can configure it to follow your process, and then you can make sure that those things are meeting FDA requirements. And if you're a younger company, you're welcome to try our platform, app dot Ketryx dot com. You can self onboard, and our templates have already passed FDA with certain companies, so you can kind of leverage that.

And, you know, for this question about next question I see here is how often are templates updated? I'm asking because the EU are always changing things, so templates are updated with the changes and regulations, and the companies can update them themselves. It's a great question. Okay. Wonderful.

Thank you all for the time, and I wish everybody a happy holiday season.