SciGroveBeta
Machine Learning

AISPA: User-Centric System Prompt Auditing for Large Language Model Applications

Xiangning Lin, Shenzhe Zhu, et al.

Featured August 3, 2026

Original Paper

This analysis was generated by SciGrove. Upload your own PDFs or enter a DOI — and get the same AI breakdown on any paper.

Get started

AI-generated analysis — This is SciGrove's AI interpretation of the paper, not peer-reviewed content. Always refer to the original paper.

Simply

A new framework helps check the hidden rules (system prompts) that guide AI behavior, finding that while many rules protect users, about 40% of AI products still have rules that could work against user interests.

In depth
The paper introduces AISPA, a user-centric framework for systematically auditing system prompts in AI applications. It defines an eight-dimension taxonomy to evaluate prompt instructions as either protective or problematic, covering aspects like identity transparency, truthfulness, and user agency. The framework employs a human-in-the-loop workflow to ensure rigorous and scalable auditing, revealing pervasive problematic instructions despite a general trend towards more protective prompt designs.

Key Takeaways

  • 1
    The paper proposes AISPA, a novel user-centric framework for auditing system prompts based on an eight-dimension taxonomy derived from human rights principles.
  • 2
    The study reveals that while system prompts are becoming longer and more user-protective, approximately 40% of commercial AI products still contain at least one problematic instruction.
  • 3
    A three-round human-LLM collaborative auditing pipeline is introduced, combining LLM scalability for candidate generation with human expert judgment for precision and normative review.

Conceptual Flow

HIGH LEVEL
1
Methodology: How AISPA Audits AI Rules

The system uses a checklist of 8 user-focused areas to find good or bad instructions in an AI's hidden rules, with both AI and people helping to check.

AI's Hidden Rules
Check Against 8 Areas
Good Rules Found
Bad Rules Found
2
Results: What They Found in AI Rules

They found that AI's hidden rules are getting better at protecting users, but many still have tricky instructions, and protection varies a lot between different AI products.

Many AI Products
Compare Rule Quality
More Good Rules Over Time
Still Many Bad Rules
Big Differences Between AIs