10.1: We acknowledge that agentic AI systems can make decisions significantly faster than humans can. However, should wrong decisions be made, these errors can snowball, resulting in potentially uncontrollable hallucinatory effects.
a. How do you decide on the tradeoff between accuracy and autonomy for agentic AI systems?
đź’ˇ One of the pieces of inspiration for this topic is in defense 1. However, this topic likely arises beyond defense, too.
b. The Airbus A380 project resulted in significant cost overruns (approximately $6.1 billion). At the heart of the issue is how Airbus used two different versions of CATIA software, resulting in wiring differences.2 This case study might be related to AI in that AI models constantly improve, but the provenance of AI-generated output is not always clear in the versions and models used to generate said output. Would this be a concern in projects, and how would such an error be mitigated?
đź’ˇ In the Airbus case, one key issue arose from the differences in assumptions used to coordinate work. How is this an issue with AI-generated output? Are standard explainability methods sufficient to understand how to coordinate across different teams?
c. When OpenClaw was released, it left much food for thoughts over the extent to which AI agents could be granted permissions to carry out tasks automatically.3 This also implies autonomous decision-making. Use a threat modeling methodology such as MAESTRO to understand the extent to which OpenClaw can result in significant security threats, and comment on the extent of ability for a layperson using OpenClaw (perhaps with the help of a LLM to learn how to use it) to mitigate said threats. (Hint: Discussion 4.6 could be helpful.)
đź’ˇ Can an average user realistically defend a system that acts as both the brain and the hands? (See Discussion 4.6 for more on the principle of least privilege.)
1 Damaini, E. (2025). Autonomy and accuracy in weapon systems: The prospects of artificial intelligence | ISPI. In ISPI. https://www.ispionline.it/en/publication/autonomy-and-accuracy-in-weapon-systems-the-prospects-of-artificial-intelligence-198146
2 Airbus – A380 – Why Do Projects Fail? (n.d.). Calleam Consulting Ltd. Retrieved December 25, 2025, from Link3 OpenClaw — Personal AI Assistant. (2026). Openclaw.ai. Link
10.2: The concept of desirable difficulty (DD) was introduced to explain the process of learning.
a. Given that many use LLMs daily to instantly solve tasks, there exists little friction and opportunity to introduce DD to trigger the learning process. How do we decide whether a certain task is worth introducing DD for better learning outcomes?
đź’ˇ Think about how we etch deep memories. In what instances do DD help etch deep memories?
b. Structure the DD into tasks that you may be familiar with. An example of DD for a secure coding task could be a manual review of parts of the code before running a SAST tool. What difficulties do you find?
đź’ˇ This is meant to be a reflective question. However, if you are stuck, you can take this opportunity to think about the difficulties you face in answering this question. In some sense, this might be a meta-question.
c. Repeat (a) and (b) as if the concept of DD was targeted at younger students.
đź’ˇ If children use AI to instantly bypass the struggle of foundational math or writing, how do they develop critical basic skills? Think on how an AI can be used as a Socratic tutor, but also think of why this needs to be a supervised activity.
10.3: One example of the reviewer’s burden is at DEFCON in 2025, where a presentation was discovered to be fabricated.4 (More details on this can be found at Openwall)
a. As AI quality is likely to improve, resulting in more convincing-looking slop, how do research communities weed out slop?
đź’ˇ This is the conundrum with a huamn-in-the-loop (HITL) which we first described in Chapter 9. HITL is important to review any sort of workflow where there is AI input, but HITL can also be quickly overwhelmed from the scale of AI inputs. This is an issue in review boards which I personally faced too. There may be checks such as provenance checks, or domain expert validation. But if I cannot be an expert on every topic, what can I do?
b. Given that it takes more time to review works in such communities, how can we do so more efficiently? (No, simply saying “use AI to assess slop” is a sloppy answer.)
đź’ˇ Extending the HITL hint, we want to now create the means to help said HITL and triage what really requires HITL. But bear in mind this can also be gamed.
4 O’Flaherty, K. (2025, October 9). DEFCON talk on Linux kernel bugs drives AI slop row. The Stack. Link
10.4: The concept of “learned helplessness” can arise due to how ANI solutions exceed humans due to how ANI solutions far exceed humans at core tasks. By this line of thought, someone may feel that there is no hope in being able to exceed ANI at any task.
a. Why is this a fallacious line of reasoning?
đź’ˇ Artificial Narrow Intelligence (ANI) is a specialized tool optimized for specific, bounded metrics. What are the limits of ANI?
b. How will humans find value that ANI cannot provide?
đź’ˇ I wish I can provide you a hint for this, but I shall not. Use this as an opportunity to dig through your own thoughts. Perhaps make it a personal question. What value can you offer than an AI cannot (unless you choose to provide an AI that value).
10.5: One of the limitations highlighted by the paper cited earlier “BotSim: Mitigating the Formation of Conspiratorial Societies with Useful Bots” is that their ABM simulations assume agents consume and propagate information from their neighbors with equal probability 5. In real life, agents form communities that can both promote and hinder the propagation of different types of information.
a. Think about the following domains: scientific advice, competing geopolitical narratives and social media trends. How will the researchers’ position on “good information to counter bad through good bots” be potentially modified in such domains?
💡 How does information travel? How do we decide on what is “good information” and “bad information”? Can a bot actually tell the difference?
b. The paper also suggests testing of strategic placement of “useful bots” in these networks. In the real world, what might this take the form of?
đź’ˇ There are useful analogs of this question in the maintenance of harmony across very diverse populations. Perhaps take a leaf from that book.
5 Ng, Lynnette Hui Xian and Carley, Kathleen M. (2026) 'BotSim: Mitigating the Formation of Conspiratorial Societies with Useful Bots' Journal of Artificial Societies and Social Simulation 29 (1) 4 . doi: 10.18564/jasss.5881
10.6: Ultimately, I did not exactly answer whether AGI will come to pass. Rather, I presented a literature review on the topic of AGI before inviting you to come to your own conclusions.
a. To what extent do you agree with my own views and exposition of the AGI topic?
đź’ˇ Here, I can no longer provide any more hints. This is meant to be a deeply personal, reflective question.
b. Would your own fundamental view of humanity affect your perspectives on AI topics?
đź’ˇ Here, I can no longer provide any more hints. This is meant to be a deeply personal, reflective question.