Synthesizers

AbstractQuerySynthesizer dataclass

AbstractQuerySynthesizer(name: str = '', llm: BaseRagasLLM = llm_factory(), critic_query_prompt: PydanticPrompt = CriticUserInput(), query_modification_prompt: PydanticPrompt = ModifyUserInput(), generate_reference_prompt: PydanticPrompt = GenerateReference(), generate_user_input_prompt: PydanticPrompt = AbstractQueryFromTheme())

Bases: QuerySynthesizer

Synthesizes abstract queries which generate a theme and a set of summaries from a cluster of chunks and then generate queries based on that.

Attributes:

Name	Type	Description
generate_user_input_prompt	PydanticPrompt	The prompt used for generating the user input.

ComparativeAbstractQuerySynthesizer dataclass

ComparativeAbstractQuerySynthesizer(name: str = '', llm: BaseRagasLLM = llm_factory(), critic_query_prompt: PydanticPrompt = CriticUserInput(), query_modification_prompt: PydanticPrompt = ModifyUserInput(), generate_reference_prompt: PydanticPrompt = GenerateReference(), common_concepts_prompt: PydanticPrompt = CommonConceptsFromKeyphrases(), generate_query_prompt: PydanticPrompt = ComparativeAbstractQuery())

Bases: QuerySynthesizer

Synthesizes comparative abstract queries which generate a common concept and a set of keyphrases and summaries and then generate queries based on that.

Attributes:

Name	Type	Description
common_concepts_prompt	PydanticPrompt	The prompt used for generating common concepts.
generate_query_prompt	PydanticPrompt	The prompt used for generating the query.

BaseSynthesizer dataclass

BaseSynthesizer(name: str = '', llm: BaseRagasLLM = llm_factory())

Bases: ABC, Generic[Scenario], PromptMixin

Base class for synthesizing scenarios and samples.

QuerySynthesizer dataclass

QuerySynthesizer(name: str = '', llm: BaseRagasLLM = llm_factory(), critic_query_prompt: PydanticPrompt = CriticUserInput(), query_modification_prompt: PydanticPrompt = ModifyUserInput(), generate_reference_prompt: PydanticPrompt = GenerateReference())

Bases: BaseSynthesizer[Scenario]

Synthesizes Question-Answer pairs. Used as a base class for other query synthesizers.

Attributes:

Name	Type	Description
critic_query_prompt	PydanticPrompt	The prompt used for criticizing the query.
query_modification_prompt	PydanticPrompt	The prompt used for modifying the query.
generate_reference_prompt	PydanticPrompt	The prompt used for generating the reference.

SpecificQuerySynthesizer dataclass

SpecificQuerySynthesizer(name: str = '', llm: BaseRagasLLM = llm_factory(), critic_query_prompt: PydanticPrompt = CriticUserInput(), query_modification_prompt: PydanticPrompt = ModifyUserInput(), generate_reference_prompt: PydanticPrompt = GenerateReference(), generate_query_prompt: PydanticPrompt = SpecificQuery())

Bases: QuerySynthesizer

Synthesizes specific queries by choosing specific chunks and generating a keyphrase from them and then generating queries based on that.

Attributes:

Name	Type	Description
generate_query_prompt	PydanticPrompt	The prompt used for generating the query.

default_query_distribution

default_query_distribution(llm: BaseRagasLLM) -> QueryDistribution

Default query distribution for the test set.

By default, 25% of the queries are generated using AbstractQuerySynthesizer, 25% are generated using ComparativeAbstractQuerySynthesizer, and 50% are generated using SpecificQuerySynthesizer.

Source code in src/ragas/testset/synthesizers/__init__.py

def default_query_distribution(llm: BaseRagasLLM) -> QueryDistribution:
    """
    Default query distribution for the test set.

    By default, 25% of the queries are generated using `AbstractQuerySynthesizer`,
    25% are generated using `ComparativeAbstractQuerySynthesizer`, and 50% are
    generated using `SpecificQuerySynthesizer`.
    """
    return [
        (AbstractQuerySynthesizer(llm=llm), 0.25),
        (ComparativeAbstractQuerySynthesizer(llm=llm), 0.25),
        (SpecificQuerySynthesizer(llm=llm), 0.5),
    ]