The direction we want for bounded inference work is a Kubernetes Job. Let each task finish, then clean up its resources.
Configure time limits and automatic cleanup explicitly; completed Jobs are not always removed by default.
We want short-lived credentials and scoped network access for each task. Tools such as Envoy can help enforce that
boundary when configured for it. This adds operational overhead, but gives us more control over what an agent can access
than an unrestricted process on a personal machine.
Sources: