# Ragas

> Evaluation framework for your AI Application

Ragas is an open-source evaluation framework for LLM applications including RAG pipelines,
AI agents, and workflows. It provides objective metrics for evaluation, test data generation
capabilities, and integrations with popular LLM frameworks like LangChain and LlamaIndex.


## Getting Started

- [🚀 Get Started](https://docs.ragas.io/en/stable/getstarted/index.md)
- [Evaluate a simple LLM application](https://docs.ragas.io/en/stable/getstarted/evals/index.md)
- [Run your first experiment](https://docs.ragas.io/en/stable/getstarted/experiments_quickstart/index.md)
- [Installation](https://docs.ragas.io/en/stable/getstarted/install/index.md)
- [Quick Start](https://docs.ragas.io/en/stable/getstarted/quickstart/index.md)
- [Evaluate a simple RAG system](https://docs.ragas.io/en/stable/getstarted/rag_eval/index.md)
- [Testset Generation for RAG](https://docs.ragas.io/en/stable/getstarted/rag_testset_generation/index.md)

## Tutorials

- [Tutorials](https://docs.ragas.io/en/stable/tutorials/index.md)
- [Evaluate an AI Agent](https://docs.ragas.io/en/stable/tutorials/agent/index.md)
- [Evaluate a prompt](https://docs.ragas.io/en/stable/tutorials/prompt/index.md)
- [Evaluate a simple RAG system](https://docs.ragas.io/en/stable/tutorials/rag/index.md)
- [Evaluate an AI Workflow](https://docs.ragas.io/en/stable/tutorials/workflow/index.md)

## Core Concepts

- [📚 Core Concepts](https://docs.ragas.io/en/stable/concepts/index.md)
- [Datasets](https://docs.ragas.io/en/stable/concepts/datasets/index.md)
- [Experimentation](https://docs.ragas.io/en/stable/concepts/experimentation/index.md)
- [Components Guide](https://docs.ragas.io/en/stable/concepts/components/index.md)
- [Evaluation Dataset](https://docs.ragas.io/en/stable/concepts/components/eval_dataset/index.md)
- [Evaluation Sample](https://docs.ragas.io/en/stable/concepts/components/eval_sample/index.md)
- [Prompt](https://docs.ragas.io/en/stable/concepts/components/prompt/index.md)
- [Utilizing User Feedback](https://docs.ragas.io/en/stable/concepts/feedback/index.md)
- [Metrics](https://docs.ragas.io/en/stable/concepts/metrics/index.md)
- [List of available metrics](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/index.md)
- [Agentic or Tool use](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/agents/index.md)
- [Answer correctness](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/answer_correctness/index.md)
- [Response Relevancy](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/answer_relevance/index.md)
- [Aspect Critique](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/aspect_critic/index.md)
- [Context Entities Recall](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/context_entities_recall/index.md)
- [Context Precision](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/context_precision/index.md)
- [Context Recall](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/context_recall/index.md)
- [Factual Correctness](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/factual_correctness/index.md)
- [Faithfulness](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/faithfulness/index.md)
- [General Purpose Metrics](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/general_purpose/index.md)
- [Multi modal faithfulness](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/multi_modal_faithfulness/index.md)
- [Multi modal relevance](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/multi_modal_relevance/index.md)
- [Noise Sensitivity](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/noise_sensitivity/index.md)
- [Nvidia Metrics](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/nvidia_metrics/index.md)
- [Rubric-Based Evaluation](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/rubrics_based/index.md)
- [Semantic Similarity](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/semantic_similarity/index.md)
- [SQL](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/sql/index.md)
- [Summarization](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/summarization_score/index.md)
- [Traditional NLP Metrics](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/traditional/index.md)
- [Overview](https://docs.ragas.io/en/stable/concepts/metrics/overview/index.md)
- [Testset Generation](https://docs.ragas.io/en/stable/concepts/test_data_generation/index.md)
- [Testset Generation for Agents or Tool use cases](https://docs.ragas.io/en/stable/concepts/test_data_generation/agents/index.md)
- [Testset Generation for RAG](https://docs.ragas.io/en/stable/concepts/test_data_generation/rag/index.md)

## Metrics

- [Overview](https://docs.ragas.io/en/stable/concepts/metrics/overview/index.md)
- [List of available metrics](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/index.md)
- [Agentic or Tool use](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/agents/index.md)
- [Answer correctness](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/answer_correctness/index.md)
- [Response Relevancy](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/answer_relevance/index.md)
- [Aspect Critique](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/aspect_critic/index.md)
- [Context Entities Recall](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/context_entities_recall/index.md)
- [Context Precision](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/context_precision/index.md)
- [Context Recall](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/context_recall/index.md)
- [Factual Correctness](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/factual_correctness/index.md)
- [Faithfulness](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/faithfulness/index.md)
- [General Purpose Metrics](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/general_purpose/index.md)
- [Multi modal faithfulness](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/multi_modal_faithfulness/index.md)
- [Multi modal relevance](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/multi_modal_relevance/index.md)
- [Noise Sensitivity](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/noise_sensitivity/index.md)
- [Nvidia Metrics](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/nvidia_metrics/index.md)
- [Rubric-Based Evaluation](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/rubrics_based/index.md)
- [Semantic Similarity](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/semantic_similarity/index.md)
- [SQL](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/sql/index.md)
- [Summarization](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/summarization_score/index.md)
- [Traditional NLP Metrics](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/traditional/index.md)

## Test Data Generation

- [Testset Generation](https://docs.ragas.io/en/stable/concepts/test_data_generation/index.md)
- [Testset Generation for Agents or Tool use cases](https://docs.ragas.io/en/stable/concepts/test_data_generation/agents/index.md)
- [Testset Generation for RAG](https://docs.ragas.io/en/stable/concepts/test_data_generation/rag/index.md)

## Customization Guides

- [Customizations](https://docs.ragas.io/en/stable/howtos/customizations/index.md)
- [Caching](https://docs.ragas.io/en/stable/howtos/customizations/_caching/index.md)
- [Cancelling Tasks](https://docs.ragas.io/en/stable/howtos/customizations/cancellation/index.md)
- [Customise models](https://docs.ragas.io/en/stable/howtos/customizations/customize_models/index.md)
- [Run Config](https://docs.ragas.io/en/stable/howtos/customizations/run_config/index.md)
- [Understand Cost and Usage of Operations](https://docs.ragas.io/en/stable/howtos/customizations/metrics/_cost/index.md)
- [Adapting Metrics to Target Language](https://docs.ragas.io/en/stable/howtos/customizations/metrics/metrics_language_adaptation/index.md)
- [Modify Prompts](https://docs.ragas.io/en/stable/howtos/customizations/metrics/modifying-prompts-metrics/index.md)
- [Tracing and logging evaluations with Observability tools](https://docs.ragas.io/en/stable/howtos/customizations/metrics/tracing/index.md)
- [DSPy Optimizer](https://docs.ragas.io/en/stable/howtos/customizations/optimizers/index.md)
- [Customizing Test Data Generation](https://docs.ragas.io/en/stable/howtos/customizations/testgenerator/index.md)
- [Non-English Testset Generation](https://docs.ragas.io/en/stable/howtos/customizations/testgenerator/_language_adaptation/index.md)
- [Persona Generation](https://docs.ragas.io/en/stable/howtos/customizations/testgenerator/_persona_generator/index.md)
- [Custom Single-hop Query](https://docs.ragas.io/en/stable/howtos/customizations/testgenerator/_testgen-custom-single-hop/index.md)
- [Custom Multi-hop Query](https://docs.ragas.io/en/stable/howtos/customizations/testgenerator/_testgen-customisation/index.md)
- [Using Pre-chunked Data](https://docs.ragas.io/en/stable/howtos/customizations/testgenerator/prechunked_data/index.md)

## Application Guides

- [Applications](https://docs.ragas.io/en/stable/howtos/applications/index.md)
- [Cost Analysis](https://docs.ragas.io/en/stable/howtos/applications/_cost/index.md)
- [Adding to your CI pipeline with Pytest](https://docs.ragas.io/en/stable/howtos/applications/add_to_ci/index.md)
- [Align an LLM as a Judge](https://docs.ragas.io/en/stable/howtos/applications/align-llm-as-judge/index.md)
- [Evaluate a New LLM](https://docs.ragas.io/en/stable/howtos/applications/benchmark_llm/index.md)
- [Compare Embeddings for retriever](https://docs.ragas.io/en/stable/howtos/applications/compare_embeddings/index.md)
- [Compare LLMs using Ragas Evaluations](https://docs.ragas.io/en/stable/howtos/applications/compare_llms/index.md)
- [Evaluate and Improve a RAG App](https://docs.ragas.io/en/stable/howtos/applications/evaluate-and-improve-rag/index.md)
- [Evaluating Multi-turn Conversations](https://docs.ragas.io/en/stable/howtos/applications/evaluating_multi_turn_conversations/index.md)
- [Iterate and Improve Prompts](https://docs.ragas.io/en/stable/howtos/applications/iterate_prompt/index.md)
- [Systematic Prompt Optimization](https://docs.ragas.io/en/stable/howtos/applications/prompt_optimization/index.md)
- [Single-hop Query Testset](https://docs.ragas.io/en/stable/howtos/applications/singlehop_testset_gen/index.md)
- [Evaluate a Text-to-SQL Agent](https://docs.ragas.io/en/stable/howtos/applications/text2sql/index.md)
- [Aligning LLM Evaluators with Human Judgment](https://docs.ragas.io/en/stable/howtos/applications/vertexai_alignment/index.md)
- [Compare models provided by VertexAI on RAG-based Q&A task using Ragas metrics](https://docs.ragas.io/en/stable/howtos/applications/vertexai_model_comparision/index.md)
- [Evaluations with Vertex AI models](https://docs.ragas.io/en/stable/howtos/applications/vertexai_x_ragas/index.md)

## CLI

- [Ragas CLI](https://docs.ragas.io/en/stable/howtos/cli/index.md)
- [Agent Evaluation Quickstart](https://docs.ragas.io/en/stable/howtos/cli/agent_evals/index.md)
- [LLM Benchmarking Quickstart](https://docs.ragas.io/en/stable/howtos/cli/benchmark_llm/index.md)
- [Improve RAG](https://docs.ragas.io/en/stable/howtos/cli/improve_rag/index.md)
- [Judge Alignment Quickstart](https://docs.ragas.io/en/stable/howtos/cli/judge_alignment/index.md)
- [LlamaIndex Agent Evaluation Quickstart](https://docs.ragas.io/en/stable/howtos/cli/llamaIndex_agent_evals/index.md)
- [Prompt Evaluation Quickstart](https://docs.ragas.io/en/stable/howtos/cli/prompt_evals/index.md)
- [RAG Evaluation](https://docs.ragas.io/en/stable/howtos/cli/rag_eval/index.md)
- [Text-to-SQL Evaluation Quickstart](https://docs.ragas.io/en/stable/howtos/cli/text2sql/index.md)
- [Workflow Evaluation Quickstart](https://docs.ragas.io/en/stable/howtos/cli/workflow_eval/index.md)

## Integrations

- [Integrations](https://docs.ragas.io/en/stable/howtos/integrations/index.md)
- [AG-UI Integration](https://docs.ragas.io/en/stable/howtos/integrations/_ag_ui/index.md)
- [Arize](https://docs.ragas.io/en/stable/howtos/integrations/_arize/index.md)
- [Athina AI](https://docs.ragas.io/en/stable/howtos/integrations/_athina/index.md)
- [Haystack Integration](https://docs.ragas.io/en/stable/howtos/integrations/_haystack/index.md)
- [Helicone](https://docs.ragas.io/en/stable/howtos/integrations/_helicone/index.md)
- [Langchain](https://docs.ragas.io/en/stable/howtos/integrations/_langchain/index.md)
- [Langfuse](https://docs.ragas.io/en/stable/howtos/integrations/_langfuse/index.md)
- [LangGraph](https://docs.ragas.io/en/stable/howtos/integrations/_langgraph_agent_evaluation/index.md)
- [Langsmith](https://docs.ragas.io/en/stable/howtos/integrations/_langsmith/index.md)
- [LlamaIndex](https://docs.ragas.io/en/stable/howtos/integrations/_llamaindex/index.md)
- [Openlayer](https://docs.ragas.io/en/stable/howtos/integrations/_openlayer/index.md)
- [Comet Opik](https://docs.ragas.io/en/stable/howtos/integrations/_opik/index.md)
- [Tonic Validate](https://docs.ragas.io/en/stable/howtos/integrations/_tonic-validate/index.md)
- [Zeno](https://docs.ragas.io/en/stable/howtos/integrations/_zeno/index.md)
- [AG-UI](https://docs.ragas.io/en/stable/howtos/integrations/ag_ui/index.md)
- [Amazon Bedrock](https://docs.ragas.io/en/stable/howtos/integrations/amazon_bedrock/index.md)
- [Google Gemini](https://docs.ragas.io/en/stable/howtos/integrations/gemini/index.md)
- [Griptape](https://docs.ragas.io/en/stable/howtos/integrations/griptape/index.md)
- [Haystack](https://docs.ragas.io/en/stable/howtos/integrations/haystack/index.md)
- [LangChain](https://docs.ragas.io/en/stable/howtos/integrations/langchain/index.md)
- [LangSmith](https://docs.ragas.io/en/stable/howtos/integrations/langsmith/index.md)
- [LlamaStack](https://docs.ragas.io/en/stable/howtos/integrations/llama_stack/index.md)
- [LlamaIndex Agents](https://docs.ragas.io/en/stable/howtos/integrations/llamaindex_agents/index.md)
- [OCI Gen AI](https://docs.ragas.io/en/stable/howtos/integrations/oci_genai/index.md)
- [R2R](https://docs.ragas.io/en/stable/howtos/integrations/r2r/index.md)
- [Swarm](https://docs.ragas.io/en/stable/howtos/integrations/swarm_agent_evaluation/index.md)

## API Reference

- [API References](https://docs.ragas.io/en/stable/references/index.md)
- [aevaluate()](https://docs.ragas.io/en/stable/references/aevaluate/index.md)
- [Cache](https://docs.ragas.io/en/stable/references/cache/index.md)
- [Embeddings](https://docs.ragas.io/en/stable/references/embeddings/index.md)
- [evaluate()](https://docs.ragas.io/en/stable/references/evaluate/index.md)
- [Schemas](https://docs.ragas.io/en/stable/references/evaluation_schema/index.md)
- [Executor](https://docs.ragas.io/en/stable/references/executor/index.md)
- [Generation](https://docs.ragas.io/en/stable/references/generate/index.md)
- [Graph](https://docs.ragas.io/en/stable/references/graph/index.md)
- [Integrations](https://docs.ragas.io/en/stable/references/integrations/index.md)
- [LLMs](https://docs.ragas.io/en/stable/references/llms/index.md)
- [Metrics](https://docs.ragas.io/en/stable/references/metrics/index.md)
- [Optimizers](https://docs.ragas.io/en/stable/references/optimizers/index.md)
- [Prompt](https://docs.ragas.io/en/stable/references/prompt/index.md)
- [RunConfig](https://docs.ragas.io/en/stable/references/run_config/index.md)
- [Synthesizers](https://docs.ragas.io/en/stable/references/synthesizers/index.md)
- [Schemas](https://docs.ragas.io/en/stable/references/testset_schema/index.md)
- [Tokenizers](https://docs.ragas.io/en/stable/references/tokenizers/index.md)
- [Transforms](https://docs.ragas.io/en/stable/references/transforms/index.md)