Schemas
TestsetSample
Bases: BaseSample
Represents a sample in a test set.
Attributes:
Name | Type | Description |
---|---|---|
eval_sample |
Union[SingleTurnSample, MultiTurnSample]
|
The evaluation sample, which can be either a single-turn or multi-turn sample. |
synthesizer_name |
str
|
The name of the synthesizer used to generate this sample. |
Testset
Bases: RagasDataset[TestsetSample]
Represents a test set containing multiple test samples.
Attributes:
Name | Type | Description |
---|---|---|
samples |
List[TestsetSample]
|
A list of TestsetSample objects representing the samples in the test set. |
to_evaluation_dataset
to_evaluation_dataset() -> EvaluationDataset
Converts the Testset to an EvaluationDataset.
to_list
Converts the Testset to a list of dictionaries.
Source code in src/ragas/testset/synthesizers/testset_schema.py
from_list
classmethod
from_list(data: List[Dict]) -> Testset
Converts a list of dictionaries to a Testset.
Source code in src/ragas/testset/synthesizers/testset_schema.py
QueryLength
Bases: str
, Enum
Enumeration of query lengths. Available options are: LONG, MEDIUM, SHORT
QueryStyle
Bases: str
, Enum
Enumeration of query styles. Available options are: MISSPELLED, PERFECT_GRAMMAR, POOR_GRAMMAR, WEB_SEARCH_LIKE
BaseScenario
Bases: BaseModel
Base class for representing a scenario for generating test samples.
Attributes:
Name | Type | Description |
---|---|---|
nodes |
List[Node]
|
List of nodes involved in the scenario. |
style |
QueryStyle
|
The style of the query. |
length |
QueryLength
|
The length of the query. |
SpecificQueryScenario
Bases: BaseScenario
Represents a scenario for generating specific queries. Also inherits attributes from BaseScenario.
Attributes:
Name | Type | Description |
---|---|---|
keyphrase |
str
|
The keyphrase of the specific query scenario. |
AbstractQueryScenario
Bases: BaseScenario
Represents a scenario for generating abstract queries. Also inherits attributes from BaseScenario.
Attributes:
Name | Type | Description |
---|---|---|
theme |
str
|
The theme of the abstract query scenario. |