Prompt-Induced Bias in LLM Security Recommendations: A Quantitative Behavioral Study
DOI:
https://doi.org/10.71465/csb222Keywords:
Prompt Bias, Security Recommendation, Behavioral Analysis, Risk Framing, Llm Decision PatternsAbstract
This study explores how prompt phrasing influences the securityrecommendations produced by large language models. Using 4,200controlled prompt variations,we analyze how prescriptive wording, threat emphasis, risk framing, and confidence cues shape LLM suggestions related to input validation, authentication, and cryptographic configurations. Results reveal substantial prompt-induced bias: risk-framing prompts increase conservative recommendations by 31%, while permissive wording reduces strict security advice by 23%. Moreover, confidence-leading prompts cause over-assertive but incorrect recommendations in 12% of cases. A mitigation framework based on neutralization templates reduces bias variance by 64%. These findings highlight the importance of prompt neutrality for dependable security decision support.
References
Mohsin, A., Janicke, H., Wood, A., Sarker, I. H., Maglaras, L., & Janjua, N. (2024). Can we trust large language models generated code? a framework for in-context learning, security patterns, and code evaluations across diverse llms. arXiv preprint arXiv:2406.12513.
Yang, M., Wang, Y., Shi, J., & Tong, L. (2025). Reinforcement Learning Based Multi-Stage Ad Sorting and Personalized Recommendation System Design.
Clement, M. (2025). Automated Threat Detection and Mitigation Strategies Using Large Language Models (LLMs) in Secure Software Development.
Iyenghar, P. (2025). Empirical Evaluation of Reasoning LLMs in Machinery Functional Safety Risk Assessment and the Limits of Anthropomorphized Reasoning. Electronics, 14(18), 3624.
Peng, H., Jin, X., Huang, Q., & Liu, S. (2025). E-commerce Intelligent Recommendation Optimization and Personalized Marketing Strategy Based on Big Model.
Bai, W., Xuan, K., Huang, P., Wu, Q., Wen, J., Wu, J., & Lu, K. (2024). Apilot: Navigating large language models to generate secure code by sidestepping outdated api pitfalls. arXiv preprint arXiv:2409.16526.
Geroimenko, V. (2025). The Essential Guide to Prompt Engineering: Key Principles, Techniques, Challenges, and Security Risks. Springer Nature.
Du, Y. (2025). Research on Digital Quality Traceability System for Temperature-Controlled Supply Chain of Foreign Trade Wine Driven by Blockchain and IoT. Business and Social Sciences Proceedings, 4, 57-65.
Das, B. C., Amini, M. H., & Wu, Y. (2025). Security and privacy challenges of large language models: A survey. ACM Computing Surveys, 57(6), 1-39.
Dorison, C. A., & Heller, B. H. (2022). Observers penalize decision makers whose risk preferences are unaffected by loss–gain framing. Journal of Experimental Psychology: General, 151(9), 2043.
Mao, Y., Chang, K. M., & Chen, Z. (2026). Research on Frontend-Backend Collaboration and Performance Optimization for High-Concurrency Web Systems.
Pawitan, Y., & Holmes, C. (2025). Confidence in the reasoning of large language models. Harvard Data Science Review, 7(1), 2644-2353.
Albassami, Z., Algarni, A., Qahmash, A., & Ahmad, Z. (2025). A comprehensive review of AI-driven Q&A systems with taxonomy, prospects, and challenges. Knowledge and Information Systems, 1-24.
Mao, Y., Chen, Z., & Ma, X. (2026). Research on a Lightweight Full-Stack Edge Execution Optimization Framework Based on Serverless and WebAssembly.
Mustafa, A. B., Ye, Z., Lu, Y., Pound, M. P., & Gowda, S. N. (2025). Anyone can jailbreak: Prompt-based attacks on LLMs and T2Is. arXiv preprint arXiv:2507.21820.
Kumar, A., Murthy, S. V., Singh, S., & Ragupathy, S. (2024). The ethics of interaction: Mitigating security threats in llms. arXiv preprint arXiv:2401.12273.
Du, Y. (2025). Research on Deep Learning Models for Forecasting Cross-Border Trade Demand Driven by Multi-Source Time-Series Data. Journal of Science, Innovation & Social Impact, 1(2), 63-70.
Murray, C. J. (2024). Findings from the global burden of disease study 2021. The Lancet, 403(10440), 2259-2262.
Krause, A., Kaur, H., Klemmer, J. H., Wiese, O., & Fahl, S. (2025, August). “That’s my perspective from 30 years of doing this”: An Interview Study on Practices, Experiences, and Challenges of Updating Cryptographic Code. In In 34th USENIX Security Symposium.
Hu, W. (2025, September). Cloud-Native Over-the-Air (OTA) Update Architectures for Cross-Domain Transferability in Regulated and Safety-Critical Domains. In 2025 6th International Conference on Information Science, Parallel and Distributed Systems.
Alromaih, S., Flechais, I., & Chalhoub, G. (2024). Beyond the office walls: understanding security and shadow security behaviours in a remote work context. In Twentieth Symposium on Usable Privacy and Security (SOUPS 2024) (pp. 507-525).
Liu, S., Feng, H., & Liu, X. (2025). A Study on the Mechanism of Generative Design Tools' Impact on Visual Language Reconstruction: An Interactive Analysis of Semantic Mapping and User Cognition. Authorea Preprints.
Bandi, A., Kongari, B., Naguru, R., Pasnoor, S., & Vilipala, S. V. (2025). The rise of agentic AI: A review of definitions, frameworks, architectures, applications, evaluation metrics, and challenges. Future Internet, 17(9), 404.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Liam Hartley , Emily Zhao , Sophie McMillan

This work is licensed under a Creative Commons Attribution 4.0 International License.
All articles published in the Computer Science Bulletin are distributed under an open access license, allowing unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Authors retain copyright and grant the journal the right of first publication. The journal encourages knowledge sharing while maintaining academic integrity and proper attribution.
