BaseIncomplete

CWE-1426Improper Validation of Generative AI Output

Category: other

Description

The product invokes a generative AI/ML component whose behaviors and outputs cannot be directly controlled, but the product does not validate or insufficiently validates the outputs to ensure that they align with the intended security, content, or privacy policy.

Common consequences· 1

  • Integrity — Execute Unauthorized Code or Commands, Varies by Context

Potential mitigations· 4

  • [Architecture and Design]Since the output from a generative AI component (such as an LLM) cannot be trusted, ensure that it operates in an untrusted or non-privileged space.
  • [Operation]Use "semantic comparators," which are mechanisms that provide semantic comparison to identify objects that might appear different but are semantically similar.
  • [Operation]
  • [Build and Compilation]

References

  1. https://cwe.mitre.org/data/definitions/1426.html

Related by meaning· 6

Nearest entities by semantic similarity across the cs-graph corpus.

CWE
Insecure Setting of Generative AI/ML Model Inference Parameters
CWE
Improper Neutralization of Input Used for LLM Prompting
CWE
Missing Origin Validation in WebSockets
CWE
Reliance on Cookies without Validation and Integrity Checking
CWE
Reliance on Insufficiently Trustworthy Component
CWE
Weak Authentication
Sourced from MITRE CWE 4.20. Curated for EU compliance use cases by Adam Lundqvist, Founder at SQUR.