---
title: "62304-Compliant Traceability for Automated Testing"
type: webinar-transcript
publisher: Ketryx
source: "https://fast.wistia.net/embed/iframe/onrx6hggog"
content: auto-caption transcript, proper-noun corrected
---

# 62304-Compliant Traceability for Automated Testing

*Ketryx webinar — transcript of the recorded session.*

[▶ Watch the recording](https://fast.wistia.net/embed/iframe/onrx6hggog)

---

welcome everybody to today's webinar about six two three zero four, compliance traceability for automated testing. We're gonna talk about, a bunch of different things from how, people develop kind of modern applications and and the expectation of modern development tools, to leveraging Git for sixty three or four compliant traceability. We're gonna review a live GitHub project integrated with Ketryx, and then we're gonna talk about, how to validate and how to build a validated CICD continuous integration continuous delivery pipeline. We'll have QA at the end. And just a few notes before we start.

One is, the webinar is recorded, and it'll be sent out to all of you, along with the slides following the discussion. Feel free to put questions in the q and a. We'll take some of them live. Some of them will answer asynchronously. We have a team of subject matter experts that are sitting in the back, kind of ready to respond to all the different questions.

And if you have to leave at any point, please fill out a survey. We we put at the end. It really helps us understand what content you're looking for in the next webinar. And then I placed the link kind of, on the chat or someone from my team will to, connect to our next webinar, which is how to implement PCCP, and CICD for AIML enabled devices. So thank you for joining us today, and we appreciate the time.

And we can go to the next slide. So my name is Erez Kaminski. I'm the CEO and founder of Ketryx. I used to help build developer tools at company called Wolfram Research, to build Mathematica, Wolfram Alpha, and the Wolfram Cloud. I led, kind of the field of AI ML for the medical device group at Amgen.

I used to be a scientist before that working on on plasma physics. And today, I help, other than Ketryx, do a bunch of things around the community, whether that's with Amy or Ness, to help kind of build, a better, more reliable future for patients all around the world. With me today, I'm happy to share one of our lead developers, Patrick Eker. Patrick was a a important developer in Rantastic that got sold to Adidas as well as a quite a well known independent consultant on cloud infrastructure for many different companies around the world. He started the Rescript language and is, used to be the chairman of the Rescript organization.

Again, language kind of leveraged quite heavily by Meta and other organizations. He's gonna lead the session today. And with us is also Gabriel Pasquale, our cybersecurity SME. Also spent time at Amgen around AI for quality and reliability, but spent most of his career really at MITRE, as a cybersecurity researcher and embedded system and others. So, I think you're in good hands today.

And before we get started, I just wanna mention you what is Ketryx and what we do. So Ketryx, is an automation based life cycle management system. Sometimes we call it a connected life cycle management system, and it's a solution, that is designed for life science teams and allows you to improve quality while releasing faster and reducing, safety and compliance risk. It allows you to leverage your existing developer tools like Jira, GitHub, and AWS, and many other systems now, in a way that's integrated, that's developer first, but also, provides all the needed controls and documentation, that, you know, quality professionals and regulatory professionals and regulators expect to see. And, fundamentally, what it does is allows you to enforce your procedures and the tools you're already using, automatically generate required documentation and orchestrate all of your releases.

It's the only IT system I know about that's not a medical device certified to medical device standards like ISO six two three or four and the related ISOs, because we just try to live kind of the life of of our clients who develop these complicated systems. And many, many people deploy all kinds of software and today hardware on us, everything from more traditional mobile apps to firmware, to cloud and SaaS, to, many, many different AI applications, that have gone through FDA, as well as kind of hardware, software combination systems. Always happy to answer questions about that. But I'll I'll give the floor over to Gabriel to get us started with this webinar. Wonderful.

Thank you, Erez. I really appreciate it. Well, I wanted to go ahead and and before we hand it off to to Patrick to to to facilitate the rest of the discussion and and show us how the platform works in action just to set a little context. So we're here today because of a focus on IEC sixty two thousand three hundred and four, software life cycle processes for medical device. And, here, I I really like this diagram.

It lays out each of the development activities that we go through. It shows the various deliverables or or evidence that we generate during each activity, that then go on to be subject to audits. And, you know, the main challenge, one of the main challenges that we that we deal with when running this process is how to maintain traceability of all the evidence that's generated. And today, we're talking specifically about automated testing or testing, holistically. And so I think looking at the the v model, which which is, you know, really within, this process, is helpful to to lay out the different levels of testing that we're capturing.

You know, we start at the bottom with our our work unit or our product. That could be our code in this case. We have a set of of unit unit level testing that we run on that software. And then as we move up the v to kind of our our software system specifications, we run our our functional, our, integration testing, all the way up to the top with our end to end testing where we we may do do some of our validation activities. And one of the the frustrating things that, I think developers and and r and d professionals run into is that a lot of the information, that we need to document our life cycle and a lot of the controls that we need to control aspects of our life cycle, exist in the tools that we're already using.

So on the next slide, I have a few, you know, common things that I I think come up when we talk about tools like Git and code hosting platforms like GitHub, Bitbucket, and GitLab that, you know, oftentimes you're documenting your design, aspects of your design, whether that's in code comments or you might use markup languages like Markdown, documentation frameworks like Sphinx or Doxygen, but a lot of that documentation exists within your code. When it comes to writing automated tests, your unit, functional, and end to end tests are written in code, or maybe you're using a behavioral driven development language like Cucumber with kind of human readable, human understandable, test case designs. Finally, we move on to kind of executing the automated tests. This happens within our plat within our code hosting platforms. We might even have gates to prevent code merging based upon the results of those tests.

We have functionality to support code reviews within these platforms. You know, think pull requests in GitHub, merge requests in GitLab, really powerful tools that that are disconnected from the rest of our life cycle. And then finally, being a cybersecurity professional, I like to include, you know, their cyber cybersecurity features in these platforms. When we think of GitHub and dependabot, things that are automatically making recommendations on how to reduce the number of vulnerabilities in our software, There's a whole lot of information that exists within these systems, that we need to somehow get into the rest of our life cycle and into the documentation, to show compliance with sixty two three zero four. Now if we go to the next slide, we hear a few common things, that result from systems like our development systems, Jira, GitLab, git GitHub, Bitbucket, being misaligned or decoupled from our compliance systems.

And all of the manual work that goes through keeping these two things in alignment, we hear things like, how do we, you know, effectively trace requirements and our code design, our specifications, down into the actual source code, down into our test designs that we store in our repository, as well as the results that execute in our code pipeline. We hear challenges around ensuring that our risk control measures is are tested is is highly manual. And then, kind of a newer requirement around this cybersecurity, how do we do, cybersecurity testing and ensure that that testing is traced to the other cybersecurity information like our SBOM, threat model, and so on. So with that, I'd like to jump into a short poll, where we'll kinda query the audience. What are some of the challenges you're facing with these developer tools, in streamlining your compliance for sixty two sixty two three zero four.

Once we pull in some some answers here, then we'll we'll hand it off to Patrick, and he'll jump into the platform and and cover a lot of these aspects and how we think about it at Ketryx. So let's see here. We got a few questions few answers coming in. We'll share the results shortly here. Yeah.

And here it looks like, you know, significant challenges around that traceability from requirements to specifications and tests, something we'll cover in a lot of depth today as we go through the product. And then generating that are those architecture diagrams, very time consuming. Okay. So we'll go ahead and share the results here. And with that, Patrick, I'd like to hand it off to you.

Let's jump in the platform. Thanks, Gabriel. So as you already mentioned, ISE six two three zero four requires us to maintain documentation in the form of what the standard says. It's called a configuration item or a list of configuration items. The main question here is, where should the documentation live and where is it maintained?

And this usually depends on the organization and the existing workflows and all the different teams there are and the tools they're using. But generally speaking, some configuration items have a tighter relation to each other, so it makes more sense to store them in one place. So for example, if we start at the bottom of the v model, we see the actual like, in the software, it would be the the actual software that you're writing, the code. And this code needs to be covered by unit and integration and system tests that are of automated nature. And these two belong together since you're implementing the code and then you're implementing, the test for that right away.

But also, there are other related information like software, software system specifications that are also kinda resembling the the description of what needs to be implemented. And this information, ideally, should also be stored, like, from an R and D perspective, it makes sense to store them all in the same, place, which would be the Git repository. The challenge here is that there is most likely a very high chance that there's other tools in organization used by non developers, product managers, etcetera, that are relevant for traceability. So for instance, the system requirements and the use cases are usually very high level. They are not directly related to code.

They often get created without code being existing at this point. And you also will have some kind of definition of what is the acceptance criteria in the form of validation tests. So to give proof that all the intended use, all the use cases are covered by a test. So how do we establish the traceability for all the design controls and tests and allow controlling those configuration items with a part eleven compliant electronic signature? So let's dive into Ketryx to see how our solution would look like.

And before we even dive into the product itself, I wanna show you I will set the stage of a git repository that we created for this demo. It kinda resembles an imaginary insulin dose pump application. It's software as a medical device. And in here, we have, for instance, some kind of software item or some code that resembles the dose calculation itself. So we have a Java class that says it can calculate the dose.

Now we wanna kinda keep the spec or the specification very close to this code. And the way how we can do this in Ketryx is we can use a doc documentation header to describe what the specification is for this particular code. And in a real world scenario, this would probably be a little bit more descriptive, but for the sake of this demo, it's a very short description. We also can attach certain meta information, metadata. So for instance, we can define a well defined unique item ID that we can use to refer to this item throughout our system.

But we can also like, in a case you don't provide it, it will be automatically generated as soon as Ketryx captures this information. But, usually, it's a good idea to hard code it, right away. The item title is used for the actual documentation that is being generated and the way how it is represented throughout the system, when interacting with this item. And you can also define certain relations, and that's the the interesting part of establishing traceability. So for instance, this particular specification has a parent relation to another specification.

And since we set up a demo to show you how to connect over different systems such as Jira or Git. It is, of course, also possible to define the Jira items that are tracked by Ketryx. So in this form, we have a specification that is tracked by a Jira ticket number k w nine. Now that we have the specification in place, we also want to test the specification. So we can go to our dose calculation test class that actually has a very descriptive test, of, like, verifying that our function is behaving correctly.

There might be a multitude of tests that taste, that test the same specification, of course. And in here, Ketryx is intelligent enough to understand that this Java class with an app test annotation is actually handling a test case. So this configuration item would be a test case that describes what is this test about. And this test basically tests that the dose is computed correctly. And in addition, we can, again, refer to our dose calculation.

If you can remember, we set a item ID of dose calculation for the specification, and now we refer to it with a, what we call a test relation. So now we establish traceability that this test and this particular repository is testing a particular specification item that is also tracked in code. Then we can also still apply the item title, test those calculation, and that's it. Now if we go to Ketryx, we will create we set up for this demo like this. Ketryx project that connects, particular Git repositories, can be Bitbucket, GitHub, or any arbitrary Git hosting platform you choose as long as it's, like, accessible to our platform, and also Chiro.

So don't worry about all these items that are shown here. We are just showing all the items that have been tracked for this demo purpose. The only thing we really care about right now is our dose calculation that we just created. So as soon as Ketryx picks up the changes in the form of commits, it will automatically detect the version that this complies to and displays it here for, for instance, version two. And, Patrick, I think we have one great question already, and it's does Ketryx work with Python tests as well?

We support, different programming languages, and it kinda depends on what kind of testing framework you're using. But we can definitely support a lot of different use cases. Yeah. And there are many Python, testing frameworks that we do support. So I think the general answer is, yeah, we do and extending it always.

So if there's anything, specific or specific framework, we'd we'd love to know. We have a pretty robust API that you can kinda manipulate as well. Yes. Thank you. So for the dose calculation here, we will now see that there is a certain source attached.

And if I'm looking into this, it has a unique identifier. I will, oops, I will get back, like, I get a refer like, I get referred back to the actual actual code where it was derived from and also to the relevant commit. This has been, like, tracked to this particular version in Ketryx. And also the same for the test dose calculation of course. So this one is detected as a test case and the other one is detected as a software item by the nature of documenting a certain part of code.

Now if you look at the actual record that is being created, so Ketryx picks up all the changes that are applied over time and then creates a certain record history of that. And if you're looking at this immutable record, we will see that all the relevant information is in there, such as when it was captured, what is the check sum, what is the actual item ID that is used for uniquely identifying it across the system, and also the status if it's, like, in a controlled state. In our case, we configured it in a way that it says, if it's in a closed state, then it's, like, considered controlled and electronically signed. We'll see all the informations that have been applied in the form of approvals. For instance, Gabriel has been applying a approval as an r and d lead and a quality manager.

So we can see that this is, like, verified and electronically signed. And some other interesting information such as the relations. As we saw before, we had a parent relation to a server software specification, and this one is tracked as the k w nine, which can later be shown on Chiro. We will have a look at that in a moment. And we can also see the history of all the records.

And as you can see, there has been some changes. And of course, we have been using, this project for quite some demos. So so you will see that many, many times it has been reopened into a resolved state. And later on, again, we we got it into a fully controlled state by applying an electronic signature. So we can actually also compare, how this item compares to the very first record that we tracked.

So in this case, we see in a certain diff view compared from this to this record, what has changed. And in this, it says the has parent server has been added. So in the beginning, there was no relation, and then we added this relation at this point in time. Now this is a very good way to just collect all the information, and we want to get a little bit more visual on this. So if we go to the traceability page here, we see our requirements traceability matrix.

And this is a very nice tool for getting a bird's eye view of the overall process of your development, life cycle. So if we say we are in version two dot zero, I wanna know how far are we with our design controls. Is every use case covered by design input? Is every design input covered by design outputs? In this case, we configured it in a certain way that design inputs corresponds to certain requirements.

And the design output to software item specifications. And this is, of course, the default setup that Ketryx proposes, but this can be highly configured to your own needs. It doesn't even need to be called software item spec, but can be, any kind of item name you wanted. And now to get back to our example, we had the toast, the dose calculation, and we can now see that this trace has been established. So just by defining these relations in the code, we can now pick up all the information in one very nice concise view.

And now the interesting part about automated testing is that we already have, like, automated tests set up, and this automated test needs to be executed and then reported somewhere and then, like, put together into a testing report. And as you can see here, we already have a automated test execution detected by Ketryx, and this is done by our CICD pipeline. And for that, I will need to go back to our Git repository. If we're looking back to our testing codes, we have to test in place. And, we also have a GitHub repository, so we are using GitHub actions to report all the information that we are running all the tests to Ketryx.

And this is completely now completely independent from GitHub actions. Ketryx provides certain APIs to report this information. But we also provide if you're using GitHub, we also provide a GitHub action that can be dropped into your CICD pipeline and makes it very, very easy to set it up and report your tests and all the other information you're interested in. So in this CICD pipeline, we have a test Java pipeline that sets up all the unit tests and sets up all the dependencies. And then when it runs the unit tests, it creates a tray unit XML file.

This is a very common format across different CICD tools such as Jenkins. And also generally well perceived in other programming languages, there's usually some kind of, reporting engine that allows you to render the test results in JUnit. And Ketryx is able to either pick up this JUnit XML file if you already have that or in any arbitrary format you think makes sense. Our APIs are quite flexible in that regard. So in here, we see that, the test results have been tracked.

We create an immutable record of that. We give it a certain identifier. And based on that information, we look at the commits. Or if you report it for a particular version in Ketryx, we know how to wire up this information to the relevant version. So for this particular build, we we reported a test for version two dot zero.

And then you can see that the dose calculation test has been executed in a certain test execution record. And by default, we usually take the or by default, we take the the result, what is being reported in the g unit XML file. And this is also corresponding to a certain build that has been reported. So you can also identify when it was reported by which servers or which build name and for which commit, omit Shah, which is like the particular commit this has been running the tests for. And then you can also see all the tests that are kinda related to that.

And then I think there's a a great question, Patrick. I think maybe we could this is a good time to address. I'll let you take this one. And it is what happens if a dev refactors the code, that is renames or deletes classes or deletes files entirely and uses new ones? So if our system already has tracked some information that, for instance, you have, like, the stalls calculation.

Later on, you decide this doesn't exist anymore. We still have the record, But, whenever you compare certain versions, it will tell you that this item doesn't exist anymore, and it needs to be approved that this is not existing anymore. So we will not just secretly omitting this information, but we'll make sure that the r and d leads or whoever is responsible is having a look over that and approves that this is now removed from the source code. Yeah. So it's, one is whatever happened in the past is immutable.

Whatever happened to a previous release is immutable, documentation wise, record wise, and so on and so forth. But then, right, if you delete something, you kind of need to acknowledge that, and you could also kind of obviously re recreate that connection for later versions. Yes. And then I see here another question by Thomas. Thank you, Thomas, for the question.

Is there a traceability to and from architecture grant artifacts, for example, architecture decision record or diagrams? So there is, and we also offer a very robust ways to kind of stick your architecture in the code and then automatically generate that architecture based on traceability in the code. So you can automatically kind of infer architecture diagrams from the code comments or other tickets in other systems and then create that traceability, can also be a markdown file and so on. And then you can, of course, link to them and compare to them. Hope that helps, Thomas.

Talking of markdown files, so maybe let's, take if, like, one step up. Like, we're we're currently inspecting how we would do the the first part, like, in the bottom of the v model. So we have the software specifications, the work of unit. Everything is kinda get to, tracked in Git. And now we wanna talk about how to track all these other items that are outside of the system, maybe in Chira and or how we could actually mix and match all these kind of things.

Like, sometimes you maybe have even software architects that work in Jira and define certain, software items there, but you can kinda cross refer to that as well. And, more interestingly, when we're talking about use cases, we usually want to validate the use case with an end to end test to make sure that we actually and ideally, test the application in a production like environment and really tap on the screen, tap through the application either in the form of manual tests or even, like, automated tests that all the flows have been gone through, that we actually make sure that the the insulin cannot be overdosed. And there is multiple forms how we can do that. So either we would maybe we can have a look at a test case that has been tracked on on Chira itself. And then we can also have a look at how it would look like from a Chira perspective to work in or work together with Ketryx.

So in here, we'll see that we have a test case item that is has been populated by default by Ketryx. You can define whatever item suits to you best. If you have X-ray, set up in your Chira application, we can also interact with that as well. And in here, we would describe a general description of what this test is about. We would describe objective steps that the tester needs to go through and then the expected behavior.

Or some people like to have the expected behavior right in the steps. It's up to you how to configure this. And the nice thing about this now is that we can actually see within Jira, we have a Kedricks app that you need to install, and then you will see useful widgets to help you get an overview of the whole traceability. So similar to what we have seen in the traceability graph, you will now be able to see all the items that relate to this particular test case. So we can see that this is actually a validation test for provider reviews patient data and patient and provider storage system.

And in addition, we also have the approvals widget. So at no point in time do we force the user to go to Ketryx all the time so they don't need to really learn any new tools. Instead, they will just use the Jira widget as as they're used to. They can have their own, like, configured workflows in there. For for this demo, we used the CapEx default schema.

But if you have a different setup, we can actually adapt to very complex, even multistep approval workflows. And then you can also see for this current state what approvals are missing. And in addition, if there is certain constraints you want to enforce, certain processes you want to enforce, maybe for particular test execution, you want to make sure that the test result is passed, you can configure rules that prevent you from approving the record if the test result has not been reported yet. And this is very nice and very useful. Like, if you wanna, like, work in Jira and define a test case, and then you have all the acceptance criteria and approve that.

But for instance, if you're an r and d centric team and you want to track all this information and get this well, then there is also another possibility in here. And I think Gabriel already mentioned it with behavior driven development. There is a file format which is called cucumber. It's a very, very common, industry standard format to describe end to end tests in human readable language that actually also is capable of being a non English language. So if you want to have Spanish I find this very interesting.

It's very interesting language design. So you can also have your scenarios in German or, Spanish or what whatnot. So for this one, we have a feature that's called dose, administration. It's located in the features folder in the feature file. And here we describe certain scenarios.

And for instance, here we have to test infusion limitations just to make sure that no one can use the IOS app to overdose on insulin. And, so here, we we define that iOS application is open. The insulin dose search of eight is entered, etcetera etcetera. And then in the end, we want to do certain assertions. So we have insulin is not administered, and an error has been shown that we prevented this overdose.

Now this is what we call a cucumber test, and we can also use certain meta information in a similar manner as we have seen before in our documentation headers. This is, this is also referring to certain specifications. So this is a verification test for the spectos administration and spectos reading warning. And based on this information, this is usually what you would negotiate between the product owner, what they wanna or the product team that defines what the application should be doing. And and then there is a handover to r and d that is able to to Or if you don't have, the certain capabilities because it's too hard to automate, you can also use that as instructions for manual tests.

Now we naturally pick up the feature files and via test results. So if you're reporting a test via our CICD pipeline, you can define a certain report file for Cucumber as well. And this will also be reported to, to our system. So somewhere we have our Cucumber test, which is called test infusion limitations. So we can see that here.

So it has been reported, and you can see that here we actually have a JSON description with the the whole proof that in feature steps basic, wood line x y zed, the iOS application is open. Everything is, is there. It's probably not the nicest format to look at, but, the the most important thing is that everything is documented, and this can be electronically signed. And, talking about about electronically signed, so we configured this traceability matrix to enforce items to be controlled. So as you can see, everything is green here, but there is not all items fully approved.

So there is a good way to understand and drill down how I get to a releasable state. And, this is like a a process enforcement that is very configurable, so you can actually define if we wanna turn it off for some reason or maybe there is certain controls I don't I'm not interested in. So this is very comfortable to to configure to your QMS needs. And so we are missing a approval for the KW free tickets, so we can actually head back to to our items. We can also see that these are the only ones that are on our resolve.

We can also filter by that state, and we will just first of all, we can see if there has been any changes. So here, there is a description that says adding a test here. Not probably not the best description, but you can see at least that some change has been occurred. Like, the symbol simplifies that something has changed, and we can actually review those changes. And if you're like, okay.

This looks good. Nothing has changed here too far. Made may be some metadata that's not relevant for this. We can actually approve these, two records. And since I am a very powerful user on this demo, I will not require multiple approvals from different approval groups, and it will go into a closed state right away.

And you can see that these settings will then automatically be synced to Jira, and that's basically it. So now I approved this, and everything should be now in a very nice controlled state. So So as you can see, this is all green. So what happens if we have maybe, like, a version running? There can also be parallel versions, you know, like, there's there's a lot of things going on if you're doing agile development.

So let's say we have, a bleeding edge version going on, which is called current version. And now we can see for this particular release, we even though we have all the design controls in place, we still not have all test executions done as in fully approved and, passed with a pass result. So we can also see, okay. There's something missing here. You see very clearly there is an error here.

So let's see what this is. So a result is missing. So first of all, there is no way to unintentionally lock in a version that has missing test results, and it's also very confirming that you always know at the point in time where are we with this release. Now I would say we're currently in the V and V phase where we actually run tests according to a test plan that has been approved. And this is very reaffirming to to be in control of this.

And talking of tests, if we look at we have many different views here that are relevant for for managing all of this. So in the test page, we can actually see I will just quickly touch on that. We will come to this back later to see more capabilities. But generally speaking, you can list all the tests that have been detected. So if I'm, like, introducing new tests in this particular version, for instance, in Git, they will show up here, right away.

If I'm selecting an older version, there might be less tests or even more tests if I made something obsolete. So as you can see here, we we see also all the latest test executions. We can also see if this is like a manual test execution in this case. Hey. Patrick, before we jump into that other screen, I think there's a lot of relevant questions on the screen you are just on on.

I'll just quickly view them. So one is I see this great question from Ambita. Thank you, Ambita. Is it possible to generate automated test coverage report? And, yes, it is, both as documents, but also even, just in this screen, you can kinda filter for automated tests.

Make all of them have passed. See another question, and maybe Patrick could be a better person than me to take this. It says the test inputs you show, is that a Ketryx script tool or some other tool that runs the tests? The testing I think it's some other tool. Right?

That's, like, what's, running the test. I think in this case, it's, GitHub action. Right? Yes. So we are using GitHub actions to to run, JUnit tests, for instance, for unit and integration tests.

And then we also have, like, an end to end test setup that uses Playwright for a JavaScript application and or for an iOS app. If you're using some automation tool that that runs the test, usually, you will have some kind of, report coming out of that, and this is what we send to our endpoint to to connect this information. I hope this makes sense. Like, it's really specific to the tools that you're using. It's mostly about what file do you provide to to provide this information.

So thank you, Greg, for that question. Another question I see is, is it possible to generate a test protocol from automated tests? You know, it's funny. Test protocols mean a lot of different things and a lot of different domain in pharmaceuticals. They're actually different than in medical devices, in some cases.

But I think this time, this person is asking, can you generate kind of the test case, report or this protocol of many different tests in within a document out of automated tests, then, yes, you can. And have Maybe we could touch into this because I wanted to show it anyways. So, like, now that we have all the items, we have all the automated tests, we, of course, want to get this into a format that is presentable to the authorities and to the FDA. So if you're going to the releases section here, we can see that there is now the version two dot two dot zero that has not been released yet. So this one, one dot zero, has been released already.

And here you get an actual very good, like, bird's eye view of the whole release. And you can see in the release checklist, depending on the controls you enabled, what is still missing to get to the full release. And for this particular demo, we already generated this documentation, but here you will see in the documents tab all the different documents that are relevant for, for approval. Right now, we didn't configure it to to just approve the documents, and there is, of course, more more strict rules that you can enforce. But, and we also have a default set of documents related risk management.

And maybe to give you an example for the testing report, so we will just we rendered this out. So usually you go there, and you see the testing report that can be regenerated because some things have changed in in the meantime, since we have been playing around with the datasets. So now we would look into this document here. And first of all, we make sure that all the relevant information is there such as the document version control, which is very, very important for an auditor, and the table of contents that shows you all the the test records, the test execution records, and the related test information, it relates to. So in here, we will see that the database persistence test we have been looking at has been captured in an automated test with all the relevant g unit test data, and also the cucumber, logs are traced accordingly in this testing report.

But you can also get a test plan that actually outlines the the tests that have been approved for execution in a certain release. Yeah. And just if you could stay on that test plan just for a second because there's a related question to it. And the question is, if we have a mix of manual and automated tests, does Ketryx combine the manual and automated test execution as one report? And as you can see here, it does.

Right? This is, exactly an example of that where you have both manual and automated tests, and, of course, you can apply certain formatting rules if you wanna apply for that. Yes. And it's also worth mentioning that we are providing a certain default set of documents. But if your QMS requires a different set of documents or you want to have a completely different format, because of your templates, you can also use our templating engine to access all the data and use our very, like, very simple, like, query language within the documents to insert the data as you need it.

And the last thing, I'll I'll just see a great question here. Could you share how does the stress this test traceability can be leveraged across multiple product portfolios with a shared software component. So we're talking about a system of system of variant management, which means you have many different subsystems that are shared. I don't think we'll have time to go into that in this demo. We actually have a webinar about that later this year, but we do support that in a very robust way.

And we have, many different, customers all over the world who are using that exact functionality, which is they have, a set of services that they then share across many different applications or products, that they validate independently and leverage that, kind of at higher higher levels. It's it's not just that we could do that. It's what we're designed to do. So it's a it's kind of in a very robust and unique way. Mhmm.

And I see we are getting closer to our running out of time, but so I will I will try to rep like, get to the points, really quick. So if we're doing like, in the beginning, we saw we covered section five of six zero three or four for the software development process, but we also like, if you're building a medical device, which probably many of you are interested in, you also need to apply certain risk management aspects. So section six to three or four, section seven says that you need to analyze all the hazardous situations from software. And this goes through all the stages of designing your software from the requirements to the specifications to the actual work and and validate that with certain risk controls. Or if there is a risk that needs to be assessed and it is not acceptable, you need to put some risk controls in place and make sure these are tested in every version release you you do.

And the way how we cover this in Ketryx is so we provide a framework for risk management that complies to the relevant standards for six two three zero four that they are interested in. And but if you have, like, your own system, of course, you can always use whatever we have shown you with Git based items if you wanted to keep track of it in Git or in Jira. I mean, we manage those by defaulting in Jira. But generally speaking, you have a concept of risk items, risk configuration items that can be, like, edited here. And if I am selecting this item is closed.

Okay. So we first need to get this item out of a first of all, we transition it back to resolved state so we can reopen it and make sure that this risk can be edited. So first of all, you cannot do any mistakes and edit it if it's already in a controlled state. So you need to make the conscious decision to to bring it back into an uncontrolled state. So now what we can see here is, like, we have a very basic, like, default schema that has all the information that is needed for documentation purposes.

So for instance, we have our harm that can be done to our patients and the relevant hazard hazardous situation and the sequence of events that lead to the hazardous situation. I don't want to go too much into detail. We have a initial risk analysis, which has a default framework configured. You can configure it, the way your QMS, your risk management policies work. And you can see here in a very useful widget how the, probability matrix maps over in this matrix.

And according to the calculation, you can define how a risk is assessed. And if it's not acceptable, then you need to either put a risk control in place and make sure this is, like, controlled, or you provide a benefit risk analysis. But for this case, we say we have a certain risk of insulin dosage, and we wanna have this not acceptable risk controlled with a risk control. And here, we can provide all the risk control measures that already exist in our system. That can be any kind of item.

You can put a requirement in place. You can put a software specification in place or even just a test case that verifies that this doesn't happen, like this particular risk. And as we can see, we used the insulin dose and glucose reading warning, which is most likely some kind of function that creates a dose warning. And, in here, we can also see how this, like, fulfills certain requirements, which parent's specification it has, etcetera. And I see here a few questions.

One is, can you also, trace from risk to requirements in test cases? And, yes, you can. You can use requirements, test cases, specifications as risk control measures. It's actually quite a favorable feature. You can also connect the requirement to have that coverage into your CICD pipeline in order to make sure that all the risk controls are covered by, for example, automated tests if that's possible for your application.

I see another question here. Is there a change, log for all these changes being made? So, yes, there is a change log for all the changes being made. It's exposed, and dependent on the configuration of your particular instance. Yeah.

But there is kind of very robust change log kind of that tracks everything. And then I see one last question. So that's an example of the change log right there for that particular risk. We can also track the configuration and so on. And then the last question I see is thank you, Greg.

Can you show an example of something you generate relative to architecture? So we could show that for a quick second. Just jump to the architecture area, Patrick. And you could see this. And, again, it could become more robust and more complicated, depending on your application.

It can also be related to systems to systems as we discussed earlier if you have a portfolio approach to your products. So this this kind of this toy example, very simple, architecture, but, of course, could become more and more complicated with time. And and this is not drawn, but automatically inferred, from, the traceability, basically. And this could be also done, whether in a layered form or we can have the pattern as a tree form, which allows you to also, see different risks. Patrick, if you wanna just, move it to a no.

Sorry. Just go to the architecture and move the, from layered, to tree. And then this is a more functional view where the color coding, addresses basically the level of risk of that particular device, and the layered form is more for a submission. Mhmm. Thank you for that question.

And but there is a good question about how to trace all this information between risk controls and verifying that there is a certain test applied. So what we can now see with all the information provided, we know that certain items are defined as risk controls, and we have a special marker and special support in that on our items page and on all over the pages in Ketryx. So you always see what kind of item is actually a risk control. So you can pay special attention whenever something like this has been added or changed or deleted that you rigorously revisit and see if all the tests actually covered this risk control. Because these are the ones that prevent, patients from dying or taking harm.

Now here we can see that the test infusion limitations is verifying this risk control. We can also see what parts of the architecture are actually providing certain risks or introducing certain risks. So the daily insulin dose calculation may impose certain risk, but also the dose administration is kinda related to that, which is more real because it's the actual code that can have a certain risk of providing too much insulin. So here, you can also see that, this relation exists. And and then, furthermore, for the quality management team that is responsible for creating a test plan and making sure that all the risk controls are part of this test plan.

We provide extra filters for tested items where you say, I want to test all I want to see all the tests that either test the risk control or all the tests that are actual risk controls. So you don't have to really think about it. You don't need to skim through a bunch of Excel files and existing test plans and and whatnot. You can just, like, in one button, click see. We have only one risk control here.

But, like, in real world scenarios, you probably have hundreds and and thousands of tests that you need to skim through. And with that information in place, it's very, very easy to just include this particular one for our test plan. And then so as we we select the included test, by default, everything is included, of course. But if you wanna exclude certain things because certain areas haven't been changed, you can also chat, you can also filter for all the test cases that are relevant for testing changed items. So we can see in this particular release, we have, like, these three tests to test an item that has been detected to be changed.

And then we can just, like, either if we use a certain item, and then we we exclude this one because the feedback form has not been touched and we don't want to test it right away, then we can actually do that and also approve the test plan with an electronic signature. Here, I'm using an OTP token, but you can also have biometric authentication. So this is essentially the this will cover all the needs for section seven of risk management so you can do your typical software development process plus make sure that your developers are actually following all the the good practice of managing all the risks. And another section is very relevant is soup management. So we need to keep track, like, in a similar manner as we do with risks.

We need to keep track of all the third party dependencies off the shelf software that we are using in our application and make sure that all the vulnerabilities have been accounted, all the cybersecurity risks have been applied and assessed. And the way we do this is via our SBOM module, which by default captures all information from your Git repository. We, support a bunch of, different dependency files from Python, from, like, requirements dot TXT file or from JavaScript of package log files or Gradle files for Java. And here we can, for instance, see there is a package log for a JavaScript application. So we see that there is a bunch of dependencies in there with a certain range, like a certain version range that has been installed.

And on every, Git synchronization, it will pick up these changes and then present to you a list of all like, a flat list of all the projects that have been connected with all the dependencies that need to be assessed. And the same thing, like, the same life, like, workflow goes for these dependency items. And if you look into this information, we can see that we automatically keep track of all the meta information we find in these files. And in addition, you can also, like, modify or or add certain information that's relevant to you and relevant for the assessment. So you can also define a certain range if you're an organization that says, well, for this particular dependency, we want to have, like, older versions, which is probably not recommended, but maybe a version range between one dot four and one dot eight or something.

Then you can actually set this custom rule sets as well and then approve that. You can apply certain risk level and all the requirement connections and that are basically relying on this used, like, dependency. And we'll see this in a minute. And so for instance, if we are looking at like, we have, like, one software item that uses next, which is a front end and back end software, back end framework for JavaScript. We have a server component that we already saw, k w nine, that is related to this dependency.

So it's very easy to to find the connections where this dependency is being used, then we can assess if this is actually imposing a risk and and then accept it and make sure that everything is in place for our final SBOM document. For vulnerabilities, we also tap into well known public repositories for software vulnerabilities and compare that with the dependencies and versions that that have been installed in your applications and give you, like, warnings if there are certain severity, like, vulnerabilities with a certain severity. And then you can also, again, go in there. You can do an impact assessment and say, well, we're using jQuery in a very old version, but we are not using the particular function that causes this cross site scripting, cybersecurity issue. So we are fine with that too, that it's not relevant to us.

So we can make this assessment here and then put our signatures on top of that, but we can also connect certain risks to that. If there is a certain vulnerability that imply that imposes certain risk we need to take care of, we can do that. And if there's, like, a similar manner as a risk control, we can also connect mitigations that might be, like, certain requirement that make sure that this doesn't cause any issues or even better, a softer item. And then we can since we are changing this record, now it's again back into a resolved state so we can, again, approve it for an electronic signature and go along our ways when everything is controlled. This is like a default setup, so we will, of course, like, get this information from Git.

But if you have different tools that you wanna integrate because you're already using, like, for instance, Snyk that that generates an s bam report for you. It has all the information. You can also use industry standard formats and report them in a similar manner as our tests. Like, if you're reporting tests, you can also provide certain SPDX files and, also later to see your CycloneDX files, that will automatically be applied to a certain version again. And then you will also see this kind of information in there and then adjust it accordingly if there's anything missing, that you need for your approval.

That covers the s pump part, and we are running out of time, unfortunately. So the the only thing I would just wanted to mention is that this was the the typical release, like, the full release process in Ketryx where we actually release a full version with all its requirements and all specifications. But we also have, so this already helped customers to drastically reduce the release cycle time. So, in fact, we'll publish a case study next week about that or reasonable amount of time. One of our customers was able to cut their release cycle time in half actually to achieve releases every two weeks and all that over a course of six months.

So we are really excited that this helps them to get the software out very, very quickly. And we I mean, this was the normal release cycle, but we actually think that there is actually one step we can go further as in we want to establish proper validated CICD in a sense that every pull request that you create may be a change into your system, and you actually want to have an incremental release that creates all the like, it's a a mini release that creates all the evidence documents, all the electronic signatures, and that actually allows you to also block merging a pull request until all the formality has been done with all the electronic signatures. And as soon as you release it on CapTrack, it ungates you to actually merge the pull request into your code. And, therefore, you're always sure that you actually release quick changes without, violating processes. Alright.

This wraps up the demo. Ketryx allows you to extract all the just to recap, the Ketryx extracts all the information you need from the different systems without leaving your actual tools. You can also use markdown files and cucumber files and and document in code and keep track of that with electronic signatures. We established the traceability across that to give you a good overview of the process, and we also record all the automated tests via CICD builds and maintain all the release documents you need to present to the auditors and have part eleven compliant signatures. That was a very lengthy demo, and, thank you for attention, and that's it.

Thank you. Thank you very much, Patrick. That was wonderful. Appreciate your time. I appreciate the audience's time.

I know this is quite complex. So I just wanted to share just the last few words. I think we're it'll be on the on the next slides. I I wanted to thank everybody for coming to talk to us today and listening to us. I wanna tell you that this is just part of a journey.

And in this journey, we're gonna integrate all software and product development tools into one one system that allows companies to deliver, more value safer value to patients faster while still meeting the same regulatory requirements with a higher level of quality assurance, and safety. And, many of our customers feel this exact same way that they are producing safer products faster, with less overhead. And, you know, we're seeing numbers that are quite shocking, like reducing the release cycle by half and more. So I'm really, really excited, about everything that's going on. Thank you, Patrick and Gabriel for doing this.

And I just wanna say that the team at Ketryx is here to help you, use your preferred tools to meet six two three or four. There are gonna be links in the chat now for our upcoming webinar in just a few weeks that I'll be hosting about implementing PCCP and CICD for AI and ML, enabled medical devices. It's related to a topic I gave a few months ago at MedCon that was very, very well attended. I'm excited to share it with you, and we'll go through a live example of how do you build a system of systems, including fast release cycles that has a machine learning enabled part that can deterministically follow a predetermined change control plan with a level of control and documentation and testing that the FDA and other regional regulators expect and how you can do that even daily or hourly if you'd like to. So I I would love to see you in that talk.

And I thank you all for attending. There's patients all around the world. They depend on people like you, to get them products faster, safer, and allow them to live a normal life like we all want to. So thank you for the time, and, we'll stay here for some q and a. And you're always welcome to reach out to us at info at Ketryx dot com, to ask more questions or to come see a demo of the product.

This is just a sliver of functionality compared to everything you can do and everything we're building this year to do. So thank you.