
by Giskard
Giskard is an open-source Python library for testing and evaluating AI agents. It covers eval checks, LLM-as-judge scoring and automated red teaming.
Web appFreemium
Step 1: Install with pip install giskard (Python 3.12 or later is required).
Step 2: Add a provider extra such as giskard[openai] and set its API key if you use LLM judges.
Step 3: Wrap your model or agent as a callable target.
Step 4: Build a Scenario with interact and check steps, then call run_sync or await run.
Step 5: Install giskard[scan] and run vulnerability_scan with a description of your agent to generate adversarial tests.
Step 6: Read the printed report and fix failing behavior.
It is a fresh rewrite for dynamic, multi-turn testing of agents, with fewer heavy dependencies. v2 remains available but is not actively maintained.
Yes. Judges and scan generators need a provider extra installed and the matching API key.
It has optional aggregated telemetry and says no prompts or outputs are sent. You can opt out with DO_NOT_TRACK=1 or GISKARD_TELEMETRY_DISABLED=1.
Yes. The README says you can wrap an ML model as a target. The automatic tabular scan remains v2-only.
WhyLabsWhyLabs uses AI for AI observability platform for monitoring model and LLM behavior, data quality, safety, and production anomalies.