Metadata-Version: 2.3
Name: athina
Version: 1.7.26
Summary: Python SDK to configure and run evaluations for your LLM-based application
Author: Shiv Sakhuja
Author-email: shiv@athina.ai
Requires-Python: >=3.9,<3.13
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Dist: RestrictedPython (>=7.1,<8.0)
Requires-Dist: bandit (>=1.7.8,<2.0.0)
Requires-Dist: chromadb-client (>=0.5.20,<0.6.0)
Requires-Dist: colorlog (>=6.7.0,<7.0.0)
Requires-Dist: datasets (>=2.16.0,<3.0.0)
Requires-Dist: editdistance (>=0.8.1,<0.9.0)
Requires-Dist: jinja2 (>=3.1.4,<4.0.0)
Requires-Dist: jsonpath-ng (>=1.6.0,<2.0.0)
Requires-Dist: jsonschema (>=4.22.0,<5.0.0)
Requires-Dist: litellm (==1.59.6)
Requires-Dist: llama-index (>=0.11.0,<0.12.0)
Requires-Dist: llama-index-vector-stores-pinecone (>=0.3.1,<0.4.0)
Requires-Dist: llama-index-vector-stores-qdrant (>=0.3.0,<0.4.0)
Requires-Dist: marvin (>=2.3.4,<3.0.0)
Requires-Dist: openai (>=1.3.4,<2.0.0)
Requires-Dist: pandas
Requires-Dist: pinecone-client (>=4.1.0,<5.0.0)
Requires-Dist: pydantic (>=2.6.3,<3.0.0)
Requires-Dist: pydantic-settings (>=2.2.1,<3.0.0)
Requires-Dist: pydantic_core (>=2.16.3,<3.0.0)
Requires-Dist: python-dotenv (>=1.0.0,<2.0.0)
Requires-Dist: pyyaml (>=6.0.1,<7.0.0)
Requires-Dist: qdrant-client (>=1.9.1,<2.0.0)
Requires-Dist: ragas (>=0.2.8,<0.3.0)
Requires-Dist: requests (>=2.31.0,<3.0.0)
Requires-Dist: retrying (>=1.3.4,<2.0.0)
Requires-Dist: textdistance (>=4.6.3,<5.0.0)
Requires-Dist: textstat (>=0.7.4,<0.8.0)
Requires-Dist: tiktoken (>=0.8.0,<0.9.0)
Requires-Dist: timeout-decorator (>=0.5.0,<0.6.0)
Requires-Dist: tokenizers (>=0.19,<0.20)
Requires-Dist: weaviate-client (>=4.7.1,<5.0.0)
Description-Content-Type: text/markdown

# Overview

Athina is an Observability and Experimentation platform for AI teams.

This SDK is an open-source repository of [50+ preset evals](https://docs.athina.ai/evals/preset-evals/overview). You can also use [custom evals](https://docs.athina.ai/evals/custom-evals/overview).

This SDK also serves as a companion to [Athina IDE](https://athina.ai/develop) where you can prototype pipelines, run experiments and evaluations, and compare datasets.

---

### Quick Start
Follow [this notebook](https://github.com/athina-ai/athina-evals/blob/main/examples/run_eval_suite.ipynb) for a quick start guide.

To get an Athina API key, sign up at https://app.athina.ai

---

### Run Evals

These evals can be run [programmatically](https://athina.ai/videos/run-evals-programmatically.mp4), or [via the UI](https://docs.athina.ai/ide/run-eval) on Athina IDE.

<img width="1530" alt="image" src="https://github.com/athina-ai/athina-evals/assets/7515552/98494736-31b6-458f-bd0a-a5b2cbca9d70">

---

### Compare datasets side-by-side ([Docs](https://docs.athina.ai/ide/compare-datasets))

Once a dataset is logged to Athina IDE, you can also compare it against another dataset.

![image](https://github.com/athina-ai/athina-evals/assets/7515552/90640acc-495e-45e0-b590-d6ddee8c5727)


Once you run evals using Athina, they will be visible in [Athina IDE](https://athina.ai/develop) where you can run experiments, evals, and compare datasets side-by-side.

---

### Preset Evals



---

### Athina Steps

To use CodeExecutionV2, you need to install e2b.

```bash
pip install e2b-code-interpreter
```

