Practical AI

Narrator: 00:01

Welcome to the Practical AI Podcast, where we break down the real world applications of artificial intelligence and how it's shaping the way we live, work, and create. Our goal is to help make AI technology practical, productive, and accessible to everyone. Whether you're a developer, business leader, or just curious about the tech behind the buzz, you're in the right place. Be sure to connect with us on LinkedIn, X, or Blue Sky to stay up to date with episode drops, behind the scenes content, and AI insights. You can learn more at practicalai.fm.

Narrator: 00:35

Now, onto the show.

Daniel: 00:41

I had the privilege of being the emcee at this year's second annual Midwest AI Summit, and it was amazing to see everyone there at the summit just engage in a really useful practical way around subjects like AI agents and decision models and AI security. And I arranged with the conference organizers for us to be able to share a few snippets from speakers at the Midwest AI Summit. So today's episode is gonna be a very special one where you'll get to hear a selection of a few snippets from speakers at the Midwest AI Summit. We'll have Heidi Porter, who is a principal at Moyer Group AI, and then Chris Shinkle, Andrew Dunlap, who are with SEP, and then Kevin Solorio, who is a staff software engineer at Focus. I hope you enjoy the content.

Daniel: 01:35

I know I certainly did. And, yeah, would love to see you at next year's Midwest AI Summit. We'll be sure to update our audience here when the details of that are shared from the conference organizers. So enjoy this selection of a few talks from the Midwest AI Summit. We have with us Heidi Porter.

Daniel: 01:56

She's a tech technology professional with decades of experience in networking, data analytics, machine learning, AI, and on premise cloud deployments at large companies. Her company, Moyer Group AI, provides education, strategy, and consulting on AI tools and where it belongs in your tasks and workflows. And it's those workflows that she's gonna talk about how to make, durable AI augmented workflows. Help me welcome Heidi Porter.

Heidi: 02:26

Alright. Thank you. Got the clicker here. Good morning. I'm Heidi Porter.

Heidi: 02:32

Quick show of hands. Who has had an AI pilot that did not make it past the demo stage or stalled at the start of production. So there's a good amount, and congratulations to all the rest of you who just, you know, hit it on the first time. Right? So a lot of times these workflows fail because...

Heidi: 02:53

It's not because the model was the problem. It's because the LM was asked to do too much. And Don and Jeff just gave great talks about when to say no to AI and when to say yes to AI. And so sometimes people ask it to hold the data, be the system of record, it's not a database, make the decision, and check its own work. The question I want you to come away with today is to look at the task in the workflow and decide what is the right mechanism for the task.

Heidi: 03:24

Okay? And then how to evaluate right answers and how to catch it when it's wrong. So evaluation, guardrails, etcetera. But before we begin to look at workflows, we need to say a word about data. So the data that goes into your workflow, I think everybody knows this a bit, but we have to talk about it.

Heidi: 03:42

It sets the ceiling on how reliable your tasks and workflows can be. And the question isn't whether your data is good, It's whether... You know, good in general. It's whether it's good enough for the task you're doing. So we check four things.

Heidi: 03:56

You know? Is it accurate complete? There's a whole list here. Is it consistent? And as Jeff talked about, maybe your agent can help with some of this.

Heidi: 04:04

But, you know, if the data is stale, it'll throw an error and just quietly give you the wrong answers. I don't know if an agent can help with that. So when there's an LLM in the loop, you need to ground it in your data at runtime, and we talk about systems of records. So you're grounding your LLM in your data. You're fetching the facts.

Heidi: 04:23

You're using the query to query a database or, some kind of knowledge graph. You're letting the evidence drive the answer, and you have its cited sources so people can check if they need to. Right? Grounding is only as good as the data behind it. So fix the data later first because everything downstream inherits its quality, right?

Heidi: 04:44

So now let's get into what a working system needs behind good, clean, integrated data. So four words we'll use for the rest of the talk. Mechanism, that's what we're going to call what actually performs a step, whether it's rules, classical ML, a decision model like JEV, and I'll talk about what that is, or an LLM. And an LLM has a large range of, you know, directly querying, you know, using an LLM to generate, using it to call tools, using it in an agentic architecture like Jeff talked about, or using it with human review. And you pick that step per task, not per project.

Heidi: 05:21

And then there's three safeguards that surround it. Right? And so you have guardrails that are runtime controls on each output. They can catch problems, escalate it to a person, constrain answers when something... You know, and then recover when something gets through anyways.

Heidi: 05:37

And then evals are repeatable tests. Evals are really important, and they're not talked about as much as they should be because they are determined whether the answer you're getting is right, whether you get a good outcome. And one of the things I want to call out is when you're looking at a workflow, don't just look at the averages because averages can hide a bad result, and you need to know how bad your bat is so you can fix it. And then observability, we won't have too much time to get into today, but Jeff talked about this a bit, being able to recover and replay and catch drift before an incident. So build each step right and then safeguard everyone.

Heidi: 06:18

And what we're gonna do today is show some examples of workflows across manufacturing, agriculture, and health care that actually worked and the lessons they learned. But first, let's look at building them. So for every task, there's a lot on this slide, I know. There are really three decisions. So the first one is what performs it, and we just talked about that here, and we talked about the fact that the LLM mechanism has a wide range of how you can implement it.

Heidi: 06:49

Second is how the work is coordinated. So if you know the steps, you probably just wanna use a fixed workflow with programming. Right? Or you can use an LM or decision model to chick... A branch at a decision point, or you can have an agentic architecture where the agent works and is constrained and bounded by your cost or time, etcetera.

Heidi: 07:12

And we'll show one example of that. And then there's third. This is where we start get... To get into guardrails. So what's allowed to execute?

Heidi: 07:21

And think of it as in a dial. Right? So if it's consequential uncertain actions, a person approves it. If it's high volume with mixed confidence, let the system act above a confidence threshold and escalate the rest or vice versa. Time sensitive work, act now and give a person a window to veto or undo it, and automate outright only what is cheap to undo.

Heidi: 07:44

So, you know, using like an agentic architecture. And whatever you pick first for these different questions, test it. Treat it as a hypothesis. And on the output of each of these tasks, you need to think about the guardrails. Right?

Heidi: 07:58

That's what we've said was very important. To make sure that you get the outcomes you're looking for, you need guardrails around every mechanism. They raise the odds of a good result, and you won't get a perfect one, so you need to measure quality against your own baselines. Okay? And the first thing to really realize, and it's been talked about a bit, is that a rule written in a prompt is not deterministically enforced.

Heidi: 08:24

A guardrail is actual machinery, a gate, a check, a human, a limit in code. And on this slide, you can see that the guardrails scale with the mechanism. And I want to call out a couple of these. One is, you know, you think, well, programming, we don't need guardrails. You should do, traditional programming has always needed guardrails, you know, not to push a bug into the code, you know, to have some change control.

Heidi: 08:49

And then another one is we we think, well, you know, if we have a human review the LLMs, and Jeff talked about this, how there'll be thousands of things to review, the human can get tired. Right? They can just have rubber stamping fatigue, and, you know, that's not a good guardrail. So you have to be really careful about these kind of things, and we've list some, you know, possible mitigations, etcetera. And then the column I want...

Heidi: 09:16

Really wanna call out is recover because something will get through eventually, and recover is how you contain it and get back to a safe state. Roll back, retract the output, undo the write, etcetera. And so you should make actions reversible where you can and where you can't require approval before they run and be careful of that human fatigue. And then I've all sit over all of this. Like I said, this often isn't talked about enough.

Heidi: 09:43

They tell you whether it's working, and they gate everything towards more autonomy. So if the valves aren't working, you don't want to promote to production because you will soon not have a working system. So I'm gonna show you some examples in real deployments that are kind of fun. So this is a published peer reviewed study. So it's not a company...

Heidi: 10:06

You know, it's not from a company. It's not marketed. It's a peer reviewed study of a rubber hose production line. And so a camera watches the line, and a modified you only look once vision model finds the defects. And if you take a look at what they're looking at, you can appreciate the difficulty of this task, especially at speed.

Heidi: 10:24

So it detects the defects, and there's no LLM in this. And then a detected defect triggers the diagnosis. The system then retrieves evidence from a knowledge graph, and it searches relevant SOPs and maintenance history. The LLM then just gets the evidence and from the systems of record. Right?

Heidi: 10:44

Not the whole knowledge base, and it drafts a candidate explanation and corrective actions. But here's the thing. Now we have this additional guardrails, right, where they use rule based checks, traditional programming, to check whether the draft stayed within the evidence that it pulled. So it's checking the LLM. And if it looks thin, if the evidence doesn't have enough support, it sends it to a human.

Heidi: 11:08

So there's two different guardrails on this. The other thing is about this example is that operators can accept, override, and comment on any of these, not just the ones that are escalated. So they have that control. So what did we learn? You know, notice what they didn't do.

Heidi: 11:25

They didn't ask the LM to look at the hose and figure everything out. The LM explains, and there's additional checks and guardrails. And the results on this are good, but they're not perfect. So, you know, grounding still needs checking. And what they did as far as observability was they monitored this for fifty two weeks to make sure it was working right.

Heidi: 11:45

So let's look at an ag example, and I like this one a lot. So I think this... Actually, I'm I'm gonna add this little note that I think this use case could be useful across a wide variety. So think about it for some of your situations. So this is a seed company called Syngenta that has a big seed catalog, and sometimes farmers select the wrong seed for their field.

Heidi: 12:08

So the salespeople or the grower bring this solution, Cropwise AI, on-site, and they do on-site seed selection based on local conditions. So the person asks the question in plain language, and then LLM interprets it. Good use of LLM. Before anything else, the user is authorized by the identity service in AWS, an application code not in the LLM. Then the Bedrock agent routes the request to knowledge bases, APIs, Python actions, data agents, and it pulls weather, soil, product, and agronomic data.

Heidi: 12:42

And when the question needs a prediction, Sedenta's existing seed and agronomy models produce it. The LLM does not do agronomy. Okay? Its second job is explain the result in plain language, and the rep or grower makes the decision. So this agent, you know, is pulling from systems of record, getting the agents it needs, and is sending out the sub agent.

Heidi: 13:04

We'll see a diagram in a second. So if you look at this, you can see blue is where the LM or agent works, gray is rules and or ML, and yellow are the guardrails. So in this case, the LM is the interface, the router, and the explainer, not the agronomist. And this is not as nice a picture as Jeff had of of the knowledge system, but you can see that, you know, there's several knowledge bases, lots of documents is pulled in. And the reason I wanted to show this kind of diagram here is because of one of the big lessons learned in this.

Heidi: 13:40

We talked about the first two already here, but the third is that the way they did not... Design that system, it doesn't just survive a model change, you know, swapping out a model. You can also swap out the knowledge bases or some of the other data sources. And so that makes that really... You know, you have to reevaluate afterwards, and, you know, if you look at the evaluation note here, they had a 100 question and answer pairs from actual sales rep, scored it by human reviewers, used an LMS judge, So they really, you know, did all these abells to make sure the workflow was successful, but it is a useful thing to know that you can swap out these different pieces pretty easily.

Heidi: 14:24

So that's our takeaway here is design so the model and knowledge bases are the easiest part of the system to replace. Now the next one I also really love is from health care. It's... Stanford Health Care built a tool called Surgical Co Management Navigator, and it screens next day elective surgeries to see which patients need surgical co management. So surgical co management, I had to look it up, is when clinicians share the practice of patient care responsibilities during pre op, inter op, and post op.

Heidi: 14:56

And before launch, routine human triage missed half of eligible patients. And so, you know, if I'm having surgery and I'm eligible for co management, think everyone in here would want that oversight there. So this system is really a big deal for them, for the patients especially. So the new system pulls the pre op anesthesia notes and the medication list from the electronic health record, and the criteria it uses are a rubric that the institution owns, written from their own literature, their own faculty expertise, and fixed in the prompt. Okay?

Heidi: 15:33

And we talked about how the prompt is, you know, readable, not enforcement. But in this case, the ELMs' only job is to apply the rubric and say yes, maybe, or no. So can't hallucinate anything. Yes, maybe, or no, that's the answers. Again, just like rubber hose, every case shows up in the EHR list, flagged or not, and the physician can, you know, look at all of them and confirm decline or override.

Heidi: 15:57

The clinician keeps the decision here. So before launch, the other thing they did was they benchmarked five different LMs against consensus chart review and picked o three minutei because it performed competitively at a lower cost. The model is again the replaceable part, not the center of the system. And what's really interesting about this case that I like is in the analysis of false negative cases, this is a small sample, but there were a minimal amount of LLM errors. What happened is they found a lot of other errors in their process, and then they were able to work to fix some of those other errors.

Heidi: 16:33

And, of course, look at the LLM errors as well. So three lessons from Stanford, and it's all about who owns the protocol. So the institution owns the criteria. They compared candidate models. And then third, they checked...

Heidi: 16:47

This is the one I want to call out is they checked why it disagreed. Right? And so what they typically did was overflag, you know, be more false positives. And when they looked at the nose, the physicians agreed 97% of the time. So it overflags by design, and physicians declined about four in 10 flagged patients basically saying it wasn't complex enough.

Heidi: 17:13

So these are three running workflows and an overview of using the LM for the right task or the orchestration piece. And so, you know, back to these four words we were talking about, the four parts we started with, we can map these to something you just saw in these examples. Right? So does each step use a simple... Simplest thing that works?

Heidi: 17:33

Syngenta used the agronomy models, guardrails, the hose line escalated when evidence was weak. The system withholds the answer and escalates. And then third, the evals. You know, we just talked about Stanford benchmarked five models. Syngenta tested against a 100 real questions.

Heidi: 17:52

And fourth, observability. We talked about the rubber hose line. They observed it for fifty two weeks. So if you can answer yes to all four of these questions, you have a system that most likely keeps working after the demo. Thank you very much.

Daniel: 18:08

Some... Thank you so much, Heidi. That was that was great. I love I love how you digged in or dug into each of those those workflows. Super practical.

Sponsor: 18:20

Whether you're an early stage startup or a Fortune 500, it's critical that you have an amazing website that looks handcrafted, but where you ship that website, updates to that website, landing pages, etcetera, in minutes, not days, to keep up with SEO and AEO and to make sure that your content is fresh. Framer is the AI website builder that helps creators, teams, and businesses ship production ready sites faster than ever while getting every detail right. Prompt, inspect, edit, and publish in one place at a whole new pace. Learn how you can get more out of your site from a Framer specialist or get started building for free today at framer.com/practicalai for 30% off a Framer Pro annual plan. That's framer.com/practicalai. Rules and restrictions apply.

Daniel: 19:23

I'm really excited for our next speaker here, Kevin Solorio. He's a staff software engineer at Focus. They're one of our our sponsors out here. I got to meet Kevin, yeah, probably a couple years ago.

Daniel: 19:40

He was he was here last year at the Midwest AI Summit. Their team at Focus is an amazing just set of AI engineers that, they build agents all day, every day. They engage with a variety of types of companies and actually get agents into production, evaluate those agents as we as we learned about this morning. Kevin is a self taught engineer and AI consultant, again, building custom agents, mainly using the the Langchains stack. Today, he's gonna be sharing with us how to build AI with workers, not for them, which is part of that earning trust on the front line.

Daniel: 20:19

So please join me in welcoming Kevin. My name is

Kevin: 20:23

Kevin Solorio. Most of this talk I am going to be speaking from is coming from one of my most recent clients at Focus. Started a new engagement, and the CEO immediately told us adoption is key. No matter what we build, adoption has to be the number one thing that happens. And we're building an app for some blue collar workers, which he upfront said are notoriously slow to adopt new technology.

Kevin: 20:50

They don't trust it. They don't like it. They don't see it. They don't want it. So in order to accomplish that, we wanted to take a few minutes and say, okay, we need to get this group of users to use a tool that they may or may not want to use.

Kevin: 21:05

So let's dive in to discover why some of these tools may have failed in the past. And so surprisingly, some of the first things that come up is it doesn't work. So when you talk to someone and they start using a tool and it doesn't meet their needs, they don't like it. They don't use it. But it doesn't work is very vague.

Kevin: 21:27

So diving into that with them, you start uncovering, well, it's slow, it crashes, it might not meet their needs. It might do the thing that it's supposed to do, but that doesn't mean it's doing the thing that they need it to. So sometimes there's just a just lack of connectivity there. The other thing that might happen is that the tool's not really helping them. Sometimes technology is adopted from several layers up in the management stream, and it's there to solve a different problem or to track a thing that they have going on.

Kevin: 21:58

And it has nothing to do with their day to day job. And if anything, it just adds more work for them to actually go into the system and update it with all the information they have to. The other thing is cumbersome. So if the tool is cumbersome to use, they're not going to, they don't want to. If they have to go into three or four different systems and extract information, and then come back to this system and put it in there, and then do that whole process, it's just not gonna be something that catches on.

Kevin: 22:27

And so that's one of the other things that happens with tools, specifically when they are built for someone. So all of these things are symptoms when you have tools that are being built on behalf of someone else. So it didn't work means typically we didn't research the requirements enough. We didn't dive into the use cases. We didn't understand exactly what we needed to build in the first place.

Kevin: 22:53

It was not built for them. If it's for someone else's, what is it, KPIs, OKRs, any of those other metrics, like if it's not really there for them, it's just in their way, they'll use it because they have to, but they might do it like at the end of the day, oh, I'll just go log everything at the end of the day, which means they've forgotten half of what's important to log. So it's not getting into your system anyway. And then the last one is the cumbersome. It's likely because it was designed by someone that wasn't doing the job.

Kevin: 23:23

They didn't understand how much of a pain it would be to go to four different systems and then go into yet another one. So if we stop building for someone, and then we actually start, I'll start going back to that a little bit more. The big thing with building for is it's always through a proxy. Most of the time, when you have a user that you're working with, like a product owner, they're interpreting the results. They're having conversations with users, but they're adding their interpretation of it, which may oversimplify things, it may take things out, it may miss very important use cases that they just gloss over.

Kevin: 24:02

It's also, again, going back to when you build for someone, you're kind of sending a signal of, I know what's best for you. This is how this work should be done, which if you're anything like my partner, she does not like it if I say, Hey, do this thing. But if I ask her nicely, no problem. So it's one of those situations where you come in and you say, do this thing, and this is how you're gonna do it. You're kinda coming back at saying, I know how to do your job better than you.

Kevin: 24:26

So it just sends a weird signal with that. And then also this tends to codify the standard operating procedures that might just be like those in theory types of cases. So in theory, all of our requests fall into one of these nice little buckets and you never go outside of that, which is rarely what happens. Contrasting that with building with someone. So when you start building with someone, you're going to start with a goal.

Kevin: 24:51

Instead of saying exactly what you're gonna do, you're gonna start by addressing what problem it is that you're really trying to solve. And by having them in the room and starting to share information across both the silos, you start to build respect and trust with each other. So I know that they know their job well. In this particular case, we were dealing with field technicians that were repairing equipment from meat grinders to ATMs. Like literally the same person being dispatched to fix your meat grinder might be the same person dispatched to fix your ATM.

Kevin: 25:23

So they have a wide breadth of knowledge. That is stuff I do not have. And when I build a system, using an AI agent that's gonna generate something, it will generate for sure. It will give me an answer. Is it right or not?

Kevin: 25:36

I have no idea. So I really need to lean on them to help correct me where I'm wrong. And this is where those in practice things come in. Now, when building with people in the day of web apps and pre AI, so like three years ago, this would largely take the form of looking at paper prototypes. You'd get in the room with them.

Kevin: 25:57

You might do some drawings on a whiteboard or some big markers and just start drawing things out. What we're really trying to define here is what information needs to be surfaced, at what time, at what fidelity. Are we getting the right information to them? Are we giving them escape patches? Is there a way for them to drill down into details that might not be there right now?

Kevin: 26:19

Now, when you start going into agents, it's less about layout because a lot of us, if you've seen anything, it's a lot of chat boxes. So there's not a lot of layout to discuss. You're gonna work with an agent. So how do you make sure that it's doing the right thing? And with the agent, your goal is the right behaviors.

Kevin: 26:39

It's calling the right tools at the right time. It's using the right guardrails. It's respecting the boundaries there. So it's not about the design as much as far as like the layouts and things like that. It's really diving into, is it doing the thing I want it to do?

Kevin: 26:55

And evals really provide the perfect scene for that. So this might be a bit of a bold statement, but an agent without evals is a proof of concept. And this doesn't matter what infrastructure it's been deployed onto. If you do not have evaluations, you cannot confirm for sure that the agent is behaving as expected. And so in this case, I'll always say that's more of a proof of concept at this phase.

Kevin: 27:26

And the nice thing is with evals, one of the nice... You can actually start testing your agent again and again and again. When I typically kick off a project like this, I will maybe synthesize a dataset where I'll take what I know about it and then create a dataset to just get something wired up and in their hands. I want someone to be able to look at it, touch it, tell me if it's working right or not, and give me that feedback. The interesting thing working with blue collar workers, they don't have traces as much.

Kevin: 28:00

If you're working with white collar workers, you might be able to get by doing most of this without truly interacting with them, though I do highly recommended having them involved. But with blue collar workers, there's not that trail. There's not the list of emails. There's not the list of traces. You don't have everything that you would normally have if you were working with a white collar worker.

Kevin: 28:22

And so to really start building for people, the first thing is to get everybody in the room, start talking about it. And going back to what I was saying earlier, that synthesized dataset, something for them to look at, I want to be wrong here. One of the things I found as a consultant is getting information from people can sometimes be very hard. But if you show them something that's wrong, they are very quick to tell you where it is incorrect. And so if you start with something knowing that, hey, this isn't gonna be right and nothing's right on the first pass.

Kevin: 28:51

So you go ahead and start with an idea. This is what I have proposed that we work on here. Let it be wrong. It's okay. Get that feedback, get that flowing in.

Kevin: 29:02

One of my early mentors used to tell me, Mary, the problem, not the solution. And that was because the problem is what's important. If you get too hung up on a solution, especially if it's one that you created, when someone comes up with a new idea that might be better or more efficient, you might feel a little hurt that your solution wasn't the one that was selected. So in this process, it's really important to realize that you're there to solve a problem and not just promote what it is that you came up with. So as you present that idea, be wrong, take the feedback and begin to iterate on it.

Kevin: 29:33

And that's where that comes together. And I feel like this is where you really start to build that trust, is when they can see how wrong it was at first. And then through some level of interaction with them, they start giving you that feedback. You can then make it so that your agent is now performing the task that you're expecting. Or it's like starting to get there.

Kevin: 29:53

The responses are making a lot more sense. With this, you are gonna start to want to get that feedback from them regularly. So in this case, you might wanna make it as easy as possible for them to give you that feedback. So if you have a dataset that you want them to label, give them a Google spreadsheet, they can go and label it on their own time, get it ready. Or if you have the time, vibe code up an app that's fun, make it gamify it, do something, get them a way to get that feedback for you.

Kevin: 30:20

And then you can use that in your evaluations. And then this is a part where you're going to iterate a lot with them, but then you might broaden the circle. So initially you might start with a handful of representatives that are going to showcase their knowledge and help you get on the right track, but they might not see every case. So then you're gonna widen that circle a little bit. So this is where you're gonna want to start creating those data flywheels.

Kevin: 30:44

If you've not used the data flywheel before, what that is, is you create an offline dataset, and that's what you're going to use for your development practices. This is what you're going to use to develop your agent against, you're going to measure against that data set. And then when you deploy it, you're going to have online evaluators that are consistently going to look at a sample of traces and kind of score them. And that could be code based evaluators, it's very deterministic, or it could be an element as judge that can then help you find and flag certain cases that were not covered by your agent before. So those that get added to an annotation queue.

Kevin: 31:20

And in that queue, you can review and discover what went wrong on that particular case. You can then add that back to your offline dataset. And now you have a constantly flowing dataset that is evolving with the use cases. So you get that flywheel set up, and then you invite more people to the platform and you start seeing how they're using it. And you rely on that data flywheel to capture those additional cases and keep building.

Kevin: 31:45

And then when you're ready, you're confident in the product, you can go ahead and release it. The big thing here, keep the loop running. You're always gonna iterate. You're gonna find cases that you didn't anticipate through the initial discovery. You're gonna find cases that you didn't anticipate through the initial release, through the first couple rounds of beta users, through the widening of that.

Kevin: 32:10

And so here is where it's really important to get that going. And then the people that participated in this, if you've ever heard of that IKEA effect, that is where you really add a lot of value to something that you help build, specifically if it went well. You know, those... That IKEA furniture that's your masterpiece in your living room, you actually prescribe a lot of value to that thing because you got your hands dirty, you built it, you were part of it. So that's one of the things that can happen with this.

Kevin: 32:38

And so then the people that you had helped build it become your champions. They're gonna really be happy with what happened. They're gonna be like, oh yeah, see this tool, I helped build this. I contributed to it. I made this better.

Kevin: 32:51

Now, one of the things with blue collar workers specifically is that they're craftspeople. They want to see that you respected them and that you also have a craft. So with this, dive in. These evaluation data sets are typically very curated. You're not gonna have millions of them.

Kevin: 33:10

You're gonna have enough that represents the use cases and make sure that you can cover all the things that you expect the agent to do. You dove into the details. There's a lot of nuance, specifically in blue collar work. It might be a thousand different things that come up and you really need to dive in and understand what made this case different from another one. And taking the time to dive into their process and understand their flow, and then be able to come back and show them that the system understands it because you took that time to say like, Hey, let me hear you out.

Kevin: 33:41

Share with me what you've learned. And also show it improving. Touching base on a frequent basis where you're able to show like, hey, I heard you, I made those changes, here's the feedback, let's keep this going. That all ends up building a really nice robust trust between the engineers and that team. And I've been fortunate enough, like right now we have a former designer, I would say former designer, he still does designing, but he's now one of our lead AI strategists.

Kevin: 34:10

And he comes in and he really listens and helps look for these nuances and helps lead that process. And it's been really great seeing how when you start working with people and really meeting where they are, that you can start building a system that everybody's really proud of and actually starts using. That is all I have, and I'm the last thing before you and lunch. So I will not keep you here for much longer.

Daniel: 34:34

That was great. Thank you so much, Kevin. I love the the... How focused this was, on on building for people. AI agents are everywhere now, and it's so easy to get access to one.

Sponsor: 34:51

However, if you're an enterprise and you're considering having this digital workforce of agents within your organization, there's a whole new level of things to consider both around human identities and agent identities. Like, let's say Bob, an employee, is onboarded into my AI platform. How do I scope his access to various models, MCP servers, tools within those MCP servers? And then if he creates an agent, can he delegate access to that same scope of things? And what if he shares that agent with Alice who has a greater level of scoping?

Sponsor: 35:31

Can she expand the scope of delegation to this agent? And what happens if Bob leaves the organization? Does that agent identity survive? These are a lot of the challenging things that are addressed if you have a powerful sovereign self hosted control layer like Prediction Guard. I'm so proud of what my team that I'm leading at Prediction Guard has built and has deployed in production at various organizations that helps you manage your fleet of agents with complete confidence, knowing what they have access to, what they have access, how their runs have developed, how they've interacted with human identities, and how you have intervened through control mechanisms in their behavior and their various runs.

Sponsor: 36:22

I encourage you to check out what we're doing at predictionguard.com/practicalai. Go

Sponsor: 36:29

ahead and schedule a demo with myself and the team to figure out how you can scale your digital workforce of agents without losing control. That's predictionguard.com/practicalai. Predictionguard.com/practicalai.

Daniel: 36:49

So this is this is, a talk from, two folks from SAP, Chris and Andrew. If you're not familiar with SEP, they've been building projects with clients across aerospace, medical, health care, finance, and and other industries.

Daniel: 37:05

So welcome Chris and Andrew from SEP.

Chris: 37:08

Click here right here. Okay. All right. Welcome. I'm sure a lot of you by now have seen the demos, the videos of people operating what they call a dark factory, hands off the keyboard, agents writing all the code, agents reviewing the code, agents reviewing the agents, agents testing the code, agents shipping the code.

Chris: 37:32

Some of that's pretty impressive and makes sense in some situations. But I'm guessing somewhere somebody in your organization has seen that demo and has asked you, Hey, when are we gonna start working like that? Like, Hey, when do we turn off the lights and kick up the dark factory? Right? So that's what we're gonna talk about today.

Chris: 37:56

And I know the picture looks like the lights are on in the factory. They're not. They're not. It's the glow of the crazy robots in there.

Andrew: 38:02

There's no humans. Yeah.

Chris: 38:04

So my name is Chris Shinkle. I'm the director of innovation at SEP, and this is Andrew Dunlap. He's one of our senior AI engineers. And today we really wanna leave you with two things. One is sort of a sense making framework, a mental model for how to make sense of how your teams should work with AI.

Chris: 38:24

We've heard talks already about how people are incorporating AI into the product. We're talking a little bit about how do you equip your teams to better make use of AI to write software. We want to give you a sense making framework. And then the second thing we really want to do is give you a set of principles that help you get started. So if you're going into a team and it's like, We wanna use more AI or we wanna do something different.

Chris: 38:47

How do you get started? Where do you go? We're gonna give you a set of principles to hopefully help with that. Now, if you've paid attention, there's lots of studies around AI. Here's three relatively recent ones.

Chris: 38:59

The 2025 door report is the oldest, but lots of studies about how people are using AI and making use of AI. And I think it's interesting that these three sort of have the same pattern, or what comes out of it is the same. And they all essentially say, AI is not gonna fix your team. It amplifies what's already there. That's really important for us.

Chris: 39:22

It's a question we have to ask when we start enabling teams with AI. See, we build software for companies where being wrong matters. Looking around a lot of regulated space, we work healthcare, pharma, aerospace. And so we think long and hard around what it looks like to enable a team with AI, and that's where the title comes from. We don't believe that every team needs a dark factory.

Chris: 39:49

We don't think you need to be all the way there to have some significant productivity gains from AI. And in fact, I would argue if you are trying to move there and you're operating in the wrong mode, even the best of teams are gonna struggle. It's gonna feel a little clunky. And so what does it look like to struggle? What does it look like that you're operating in the wrong mode?

Chris: 40:12

So you may hear something like this, We're shipping AI slop and nobody knows what right looks like. Or, We should be using more AI. Well, I don't know what more is, but we should be doing more. Or, the joy's all gone. All I do all day is review code.

Chris: 40:33

And if you've heard one of those things in the last week or been in a meeting or heard something like that, you are in the right session. When the way you work doesn't fit your situation, more AI is not gonna fix that. More AI won't fix the AI fit. And Andrew is going to walk through three examples from three different customers of ours, three different teams that are all using AI in a slightly different way or enabled with AI in in different ways. And we're gonna get into a little bit, like, why why that is.

Chris: 41:15

And we...

Andrew: 41:15

I think all three are operating where they should be. Thanks, Chris. So let's talk about a client we have, their transmission operator. And so they have these simulations, and the simulations have these models that convert from one form to another. This is a legacy system.

Andrew: 41:34

It's been used by the business for a long time. It gets the job done, but they wanna improve it. And so they had a clever engineer One day said, hey. I got these input models. I got these output models, and I got a whole bunch of agents.

Andrew: 41:46

So I can just reverse engineer them and figure out the process in the middle. Makes sense. Two days later, lo and behold, he has a working prototype. So they wanna get the prototype to prime time. Gotta get to production.

Andrew: 41:59

That's when we get the call. We bring in a team. They get started. A week passes. We haven't delivered any software yet.

Andrew: 42:08

What happened? What happened was there's new forces at play. There's some new constraints the team's facing.

Chris: 42:16

Yeah. So you're saying the constraints are what impacted the team's ability to to move as fast as we wanted?

Andrew: 42:24

Exactly. So there's six constraints that we found all our teams wanna review when they're pinpointing the forces acting on them. So we have token budget. This is a really common one. Maybe an organization is all in on AI, but how comfortable are they spending $10 a day for tokens?

Andrew: 42:42

A 100? A thousand? 10,000? That changes the conversation for the team. Organizational readiness.

Andrew: 42:50

Do you have a sponsor? Do you have someone that's willing to back this effort? Goal definition. Do we know where we're going with this? Are we trying something out?

Andrew: 42:59

Are we experimenting? Domain knowledge. How much of the domain is written down versus how much is in people's heads? That will affect the outcome when agents can play a lot. Verifiability.

Andrew: 43:12

Can you automate how... If you've gotten to write yet? Do the agents know? Do you need subject matter experts? And then code oversight.

Andrew: 43:21

This is a big one for regulated industries. If you need eyes on the code, that's gonna change how you interact with the agents. Right? So if we go back to our two day prototype, what happened? At organizational level, they had some token budget constraints.

Andrew: 43:37

When we moved to our team, they had a budget that only allowed a couple days of work before the devs ran out of tokens. At product level, there was too much domain knowledge in heads, so they weren't able to have the agents get to what the right answer was because you had to ask a person. Then at the product level, they found out, yeah, the prototype could be verified by one person, but when we scaled it to production, now we had millions of input parameters, and we didn't have a great way to scale that up to verify every output was good. So what these end up being are kinda knobs or dials that your team can can hone in on. Now some of the dials are gonna be fixed.

Andrew: 44:17

You may not have a say over token budget, but pinpointing that constraint brings up conversations that happened for our team, highlighting that these token budgets were an issue, highlight it to the organization that this is something that they need to weigh in on. For the domain knowledge, it changes who becomes interacting with the with the project. Okay. Now we need subject matter experts here because we don't have enough information to the level of automation with the agents we want. So pinpointing these constraints is the first step of what leads into the next section, which is what mode the team operates in.

Andrew: 44:54

So there's four modes that we commonly find our teams operating in. And now there may be more, there may be less, but this is a good delineation of very distinct different modes of operation, and they're spread out by a level of agent autonomy. So on the left, we have this in visual mode where there's gonna be a human in the loop. Quite often, they're gonna be doing their own thing. As we move to the right, you can see we land on the dark factory.

Andrew: 45:19

The humans have left. The lights turned off. The robots are running the place, and our products coming out the end full autonomy. So for our two day prototype, they're working in individual agentic coding mode. Right?

Andrew: 45:33

Because they were just getting a prototype out. They didn't have to worry about scaling too much. They could verify one output and say, yeah. We got a process working. Once our team scaled, that kinda broke down.

Andrew: 45:44

Right? That was bad fit. There was there was friction there now. So let's look at a different example for a different mode. So the team I'm working on, we work on some software that designs building components for a client that manufactures them.

Andrew: 46:00

So building components, this broom's actually fantastic. If you look around, we got some concrete blocks and there's some steel beams. And if you look up, there's some joists and HVAC units, so building building components. So high cost of failure. Right?

Andrew: 46:12

We don't want buildings falling down. How do you weave agents into that? Because I bet you won't be too happy if I told you, yeah, the building will stand up because our agents verified it. So we started in an individual mode, and we wanted to move to the shared mode. And to do that, we had to build a shared pile of information.

Andrew: 46:32

So what that looks like is we have domain elements, architecture. We may have workflows, rules for how to implement, code, plan, but it's all in one body of knowledge that everyone has access to. Now the the developers day to day may still kinda work on their own. They may pair with the agent, but now we know what we're doing. Right?

Andrew: 46:53

We're all... We all have the same domain rules. So the point of this is now we're giving the agents a little bit more autonomy here, and the shape of that organization differs based on the constraints that our project allowed. We could have gone further, but we didn't want to because verifiability was really important for us. Right?

Andrew: 47:12

Buildings don't fall down. That's the goal. So we weren't comfortable going further. We made that choice. This was a good fit for us.

Andrew: 47:19

Okay. What if we wanted to go further? We have a team that did that. So we have a team that worked on mobile banking app for Forum. If you're not familiar, Forum's a local credit union here.

Andrew: 47:30

So they had a banking app. It worked. It was in c sharp, and they said, hey. We wanted to go to iOS and Android native. Yeah.

Andrew: 47:39

Great. Wonderful. What about tokens? Oh, we don't care about tokens. Spend all the money you want.

Andrew: 47:44

Let's just see what we can do. Okay. So you have all the requirements set up here. Just wanna come... Go to another code base.

Andrew: 47:50

There's no cost. We have backing by the organization. So we built something that looks like what you see on the screen now. Now you can see that we're losing individuality. We're talking about a team here and a workflow of agents.

Andrew: 48:04

Agents are starting to take on roles of their own. They're talking to an orchestrator. The team is providing con... Intent, maybe some context. They're checking on the repo, but all the individually has been lost.

Andrew: 48:17

And what we saw here was we had a greater level of autonomy, and that was reflected in a six x reduction of effort for the project. So you can kinda see there how we're moving up in the steps. Right?

Chris: 48:30

Right. So the goal then, you went from individual to shared door. Like, the goal is to get to dark factory. That's what we're trying to do. Right?

Andrew: 48:37

Maybe maybe not necessarily. Because the goal here is fit. Right? That's how we started. Constraints lead to modes.

Andrew: 48:45

The mode is your fit. So what we do with our teams is we have our six constraints. We have them evaluate them. Depending on how the constraints evaluate, you may have a growing level of area that you can operate in. More constrained areas, you're gonna work in a more individualistic mode.

Andrew: 49:02

Maybe you're working on a prototype. Maybe you have a lot of constraints. Maybe you need to make sure every line of code is right. Maybe you just don't have access to the Internet. Probably doing something more individual.

Andrew: 49:13

As your constraints grow, we get that higher level autonomy like we saw with the forum team. But it's about identifying those constraints and picking the mode that best operates for you. Right?

Chris: 49:25

So you might be asking, what does this look like for me or my team? What should I be considering if I understand these constraints? How does that actually play out? I spent many years coaching teams and clients on Lean and Lean software development. It turns out Lean has an answer to this problem.

Chris: 49:44

Lean, first off, would say there's no single blueprint. There's no one right answer for everybody. Again, the reason why we don't all need a dark factory. And so Toyota never told anyone to hang an Andon cord. And you don't know what that is.

Chris: 49:57

It's a cord they would put into the factory next to a line, and anybody could pull the cord and stop the line. If they saw a problem manifested, they could stop the line, fix the problem, continue on. And that's sort of tied to Toyota's principle of build quality in. Like, don't wait till the end to inspect it, but stop, fix the problem right now, and build quality into the process as we're going. Now, interestingly enough, you may have heard of an and on cord before, may not, but Toyota did not put those on every single line in every factory.

Chris: 50:30

It was very constraint based. Based on the constraints of that situation or line, they might implement this. The principle build quality in, that applied everywhere. So principles, think about this, are true everywhere. The practices aren't.

Chris: 50:46

The practices will vary depending on, like Andrew said, your constraints. And so in twenty minutes, I can't tell you exactly what your practices should look like, but what we can do, I think, is walk through a set of principles that's gonna help you get started. Exactly. Okay. So we got five principles.

Chris: 51:10

Good news. They're easy to learn. Hard part... Hard to master, but we're

Andrew: 51:14

gonna go through them right here. So we're gonna start with the foundation, shared infrastructure. Call back to my team. We went from individual to shared. We started pulling the knowledge into the code base so everyone had access to it, that shared infrastructure.

Andrew: 51:29

Next, we have deliberate iteration. So this is a key one I see a lot of teams fall through. They are doing things. They may even have shared infrastructure, but they're not talking about it. Maybe one person is having a terrible experience with the model.

Andrew: 51:43

Maybe one person thinks the security audit on the review is just noise. Until you schedule a block of time in the week for that and talk about it, it's not going to improve. Next is making it safe, which is verification before velocity. That's exactly what happened to our team in the prototype example. Right?

Andrew: 52:02

We slowed down. That's because the scale change, the constraints change, and we needed to make sure that we were getting the right answer before we went fast. Once we're ready to go fast, next, we're looking for systematic bottlenecks. Review is gonna be a sticking point. If review is a sticking point, you need to make it not a bottleneck.

Andrew: 52:21

And once you do that, it's gonna move somewhere else. You have to keep chasing them. Remember when we called back to the research? The the team gets amplified in all their attributes. So the the bottlenecks are just gonna be worse than whatever they were before, and you're gonna feel the pain.

Andrew: 52:37

You have to keep chasing those to make the team fast. Finally, remove the toil. Script the things that happen over and over again. Save the thinking for the humans. Save the analytical processes for the LLMs.

Andrew: 52:51

It's gonna save you tokens. It's gonna be faster. It's going to just make life better for the team. Five principles. Three teams, same principles, different modes, different practices.

Andrew: 53:03

Get it? So that's it. Thank you.

Daniel: 53:05

Let's go. That was awesome. I think I think, basically, here who manages a dev team, should make that talk required listening. That that was that was awesome. And, yeah, the the the...

Daniel: 53:24

Those sorts of questions are really the questions that we're all asking as we're advancing our team and understanding how to work together, work together again.

Narrator: 53:38

All right. That's our show for this week. If you haven't checked out our website, head to practicalai.fm and be sure to connect with us on LinkedIn, X, or Blue Sky. You'll see us posting insights related to the latest AI developments, and we would love for you to join the conversation. Thanks to our partner, Prediction Guard, for providing operational support for the show.

Narrator: 53:58

Check them out at predictionguard.com. Also, thanks to Break Master Cylinder for the beats and to you for listening. That's all for now, but you'll hear from us again next week.