Frequently Asked Questions

How do you validate the agents? Do you have to validate them as they keep learning or as the model is updated?

We validate Ketryx Agents by designing test scenarios that simulate real-world use cases and cover both typical and edge-case interactions. These tests ensure the agents provide consistent, accurate, and reliable outputs. Our approach follows principles of good software validation as outlined in ISO 13485 and IEC 62304, considering completeness, traceability, and reproducibility.

Specifically, we start by defining clear requirements and acceptance criteria, selecting representative test data, and specifying expected outcomes. We execute these tests, document the results, and analyze any deviations. This validation process ensures that the AI agents behave as intended, conform to applicable internal quality and compliance standards, and maintain a high level of reliability and safety for regulated environments.

We revalidate agents when necessary, such as when a change is made to one of the underlying models, prompts, or relevant parts of the implementation.