Hi, I’m Sean, Senior AI Product Manager at JSI. Thanks for joining us for part six of our AI Essentials for Investigative Intelligence video series. Now throughout this series, we focused on the AI capabilities that truly matter for investigative work and the considerations that should guide every deployment.

Now in this episode, we turn to one of the most critical topics of all, compliance and ethics. We’ll look at what responsible AI use really means in an investigative context, why governance and transparency matter, and how organizations can adopt AI in ways that are lawful, defensible, and aligned with their mission.

Now around the world, governments are quickly grappling with the challenges of establishing legislation around the use of artificial intelligence in society and specifically within high risk environments such as law enforcement and intelligence. Now this culminated in the EU with the establishment of the EU AI Act, which is a landmark set of regulations governing how AI tools can be used within these environments.

Now this is since inspired and helped shape regulations and best practices in many other countries around the world as well.

At JSI, we take these regulations and best practices very seriously, ensuring that our AI systems are fully transparent, ethical, and keep the human in the loop for all final decision making. Now, some of the ways that we do this outside of specific model selection and testing actually relate to how we incorporate these AI models into our systems as a whole.

For instance, ensuring that things like user permissions and data access are treated as critical requirements and never exposing information to a user that they should not be permitted to see. For all AI processing or interaction on collected data, we log those interactions against each record to maintain the provenance for judicial review for any user interactions with the data or with AI systems.

They’re also fully audited. Again, allowing organizations to be transparent around how any given investigation was conducted using these tools.

And finally, to also respect the data retention requirements that are outlined within legal warrants, we ensure that any AI outputs derived from collected content are also destroyed in tandem with the original data. Now within our conversational AI tools, we’ve also included flexible alignment guardrails so that agencies can automatically detect when there may be an abuse of these systems outside of what they were intended for.

This allows system administrators or data protection officers to be alerted when a user might be acting or carrying out actions on the system that are unaligned with their organizational policies.

And when we talk about performance evaluation as well in AI enabled systems, this is not just an engineering concern from our perspective, but it’s also a governance and risk management concern for agencies that are deploying these capabilities operationally.

1st, of course, organizations should understand what AI models are actually being used within a system, which again provides information around what types of data these models were trained on and also what the known limitations or biases that may be inherent to those models that are specific to that agency’s specific use cases.

And again, that information is going to be foundational for things like internal compliance reviews when you have to answer questions around what types of models are there, what are the biases of those models, These are key questions to ask when evaluating these types of systems for operational deployment.

Now, beyond individual models, your organization should also consider how AI components behave together as part of an end to end system like we saw in the last video. Now, outputs are rarely the result of a single model.

They emerge from pipelines that include data parsing, enrichment, retrieval, and analysis. Evaluating system performance as a whole against realistic domain specific data provides a baseline for understanding both reliability and risk.

And we’ll talk a little bit about that in a second around how to run that operational testing.

And finally, performance evaluation, of course, does not end at deployment. Mature systems include mechanisms as well to capture operational metrics and end user feedback over time throughout the life cycle of that product.

This allows organizations to detect performance drift, surface unintended consequences, and again, continuously refine the usage guidance and safeguards that they have in place as part of those operational best practices.

Now finally, as I alluded to, the most important factor in deploying AI within operational or analytical workflows, especially in high risk environments, is how it’s introduced and tested in real world conditions. Now of course vendor demonstrations are useful starting points to see what technology is out there and how the tools work, but they’re not sufficient on their own.

AI capabilities really must be exercised against real world data and realistic investigative scenarios. This kind of operational testing really helps to build practical understanding of how these tools actually behave, and it helps to expose limitations, failure modes, and usage patterns that really have to be understood before these tools are relied upon in an operational environment.

These findings, of course, then need to be translated into documented best practices, whether that’s updated SOPs clear guardrails and targeted user training so that expectations and responsibilities are clearly set throughout the organization.

Now, when applied deliberately and responsibly, AI can be a powerful force multiplier against the volume and complexity of modern investigative data. However, it has to remain a support capability and not a decision authority.

Trained human professionals remain fully accountable for judgments and outcomes, with AI serving to inform, accelerate, and augment, but never to replace critical human decision making.

And with that, it wraps up our AI Essentials for Investigative Intelligence series. Really hope that this has been a practical, informative resource as you think about how and where AI can support your investigative work.

Now, if you did miss any of the earlier episodes, you’ll see them on the screen now. I do recommend watching the full series to get the most value, as each episode really does build toward a clearer, more complete picture.

If you have any questions or want to learn about how JSI can support your mission, please do visit us at jsitelecom.com. Thanks for watching and for the work that you do.