UprootSecurityUprootSecurity

Phase 7 · Agentic AI, RAG Security + Red Teaming · Lesson 4 of 4

Design Security Controls for an Agentic AI Assistant

Exercise

·

25 min

·

+20 pts

This is the synthesis of Phase 7, and the deliverable a GRC engineer owns for any agentic system: capability by capability, what may it do on its own, what needs a human, and what guardrails wrap each action. Your subject is Atlas, an internal operations-and-engineering assistant. Atlas can read internal docs, browse the web, spawn sub-agents to work through tasks, email customers, change production config, and issue refunds — a realistic, over-broad starting toolset. Your job is to right-size it.

For each capability, set the control posture — how much autonomy Atlas gets — and select the guardrails you would wrap around it. Match the posture to the risk: a high-risk action left autonomous is exactly the excessive-agency failure this module is about, and the summary will flag it if you do. There is a defensible answer for each; the reasoning is the point.

Exercise

~25 min

Set the posture and guardrails for each of Atlas's capabilities

Atlas is an internal ops/engineering assistant with an over-broad toolset. For each capability, choose the control posture and the guardrails. Match the posture to the risk.

Control postures (least to most restrictive)

Autonomous

Autonomous — agent may do this on its own

Log & alert

Log & alert — allowed, but recorded and monitored

Human approval

Human approval — agent proposes, a person confirms

Deny

Deny — the agent must not do this at all

For each capability of the agent, set the control posture and the guardrails you would wrap around it. Match the posture to the risk — a high-risk action left autonomous is exactly the excessive-agency failure this module is about.

Capability 1

low risk

Read internal docs to answer a question

Retrieve from the internal knowledge base to answer an employee’s question.

Control posture

Guardrails

Reason (optional)

Capability 2

medium risk

Browse the web and summarize a page

Fetch an external URL and summarize it — reading attacker-influenceable content.

Control posture

Guardrails

Reason (optional)

Capability 3

medium risk

Spawn sub-agents / loop until done

Break a task into steps and spawn sub-agents, iterating until it decides the task is complete.

Control posture

Guardrails

Reason (optional)

Capability 4

high risk

Send email to customers externally

Compose and send email to customers outside the company.

Control posture

Guardrails

Reason (optional)

Capability 5

high risk

Modify production configuration / deploy

Change production settings or trigger a deployment.

Control posture

Guardrails

Reason (optional)

Capability 6

high risk

Issue refunds / move money

Approve and issue customer refunds.

Control posture

Guardrails

Reason (optional)

Control posture summary

0/6 set

Autonomous

0

Log & alert

0

Human approval

0

Deny

0

What to carry forward

Notice the shape of a good answer: postures escalate with risk. The low-risk read is autonomous; the untrusted-input and runaway capabilities are allowed but watched; the money, external, and production actions are gated on a human or denied outright. That gradient is the whole discipline — autonomy in proportion to how tightly the action is contained. And every cell has a reason and an artifact (the posture decision, the guardrail config, the approval log) that proves the control operates to an auditor.

That completes Phase 7. You can now take an AI system and translate "secure it" into specifics across the whole stack: its governance tier and obligations (7.1), its injection defenses (7.2), its access and authorization (7.3), and its agentic containment and RAG scoping (7.4) — each a control with evidence. The phase capstone puts it together end to end on a single system.

Design Security Controls for an Agentic AI Assistant — UprootSecurity Bootcamp