Promptfoo

Start with the official documentation for Promptfoo and choose the setup path that matches your use case. For a hosted api, create the required account or API credentials; for an open-source project, install or deploy the supported package/runtime from its official repository.

Create a small test workflow before integrating it into a production application. Use a narrow task with known expected results so you can understand the product's configuration, model/provider requirements, tool permissions, output format, and failure behavior. If the product supports multiple models or providers, compare them on the same task rather than assuming their behavior is interchangeable.

Once the basic workflow works, connect only the data sources, tools, browser permissions, or external services that the task actually needs. Keep credentials in environment variables or a secret manager, and use the product's built-in tracing, logs, evaluation, or usage controls where available. For agentic tools, test tool-call boundaries and recovery behavior before allowing broader autonomy.

Finally, validate cost, latency, reliability, and data-handling requirements under realistic traffic. Review the official documentation again before launch because APIs, model support, pricing, and deployment instructions can change. For open-source software, pin a tested release and review its license and upgrade notes.