Metadata-Version: 2.3
Name: aibrix
Version: 0.4.0rc1
Summary: AIBrix, the foundational building blocks for constructing your own GenAI inference infrastructure.
License: Apache-2.0
Author: AIBrix Authors
Author-email: brosoul@126.com
Requires-Python: >=3.10,<3.13
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Dist: aiohttp (>=3.11.7,<4.0.0)
Requires-Dist: boto3 (>=1.35.5,<2.0.0)
Requires-Dist: dash (>=2.18.2,<3.0.0)
Requires-Dist: fastapi (>=0.112.2,<0.113.0)
Requires-Dist: filelock (>=3.16.1,<4.0.0)
Requires-Dist: gunicorn (>=23.0.0,<24.0.0)
Requires-Dist: hf-transfer (>=0.1.8,<0.2.0)
Requires-Dist: httpx (>=0.27.2,<0.28.0)
Requires-Dist: huggingface-hub (>=0.24.6,<0.25.0)
Requires-Dist: incdbscan (>=0.1.0,<0.2.0)
Requires-Dist: kubernetes (>=31.0.0,<32.0.0)
Requires-Dist: matplotlib (>=3.9.2,<4.0.0)
Requires-Dist: numpy (==1.26.4)
Requires-Dist: pandas (>=2.2.3,<3.0.0)
Requires-Dist: prometheus-client (>=0.20.0,<0.21.0)
Requires-Dist: pulp (==2.8.0)
Requires-Dist: redis (>=5.2.0,<6.0.0)
Requires-Dist: tiktoken (>=0.7.0,<0.8.0)
Requires-Dist: tos (==2.8.0)
Requires-Dist: transformers (>=4.38.0)
Requires-Dist: types-redis (>=4.6.0.20241004,<5.0.0.0)
Requires-Dist: types-requests (>=2.31.0,<3.0.0)
Requires-Dist: uvicorn (>=0.30.6,<0.31.0)
Project-URL: Repository, https://github.com/vllm-project/aibrix/tree/main/python/aibrix
Description-Content-Type: text/markdown

# AI Runtime
A versatile sidecar enabling metric standardization, model downloading, and management.

## Quick Start
### Installation
AI Runtime can be installed by `pip`.

```sh
pip install aibrix
```

### Model download
The AI Runtime supports model downloading from the following storage backends:
* HuggingFace
* S3
* TOS

For more details on model downloading, please refer to our [Runtime docs](https://github.com/vllm-project/aibrix/blob/main/docs/source/features/runtime.rst#model-downloading).

### Integrate with inference engines
The AI Runtime hides various implementation details on the inference engine side, providing a universal method to guide model management, as well as expose inference monitoring metrics.

At present, `vLLM` engine is supported, and in the future, `SGLang` and other inference engines will be supported.

For more details on integration with `vLLM`, please refer to our [Runtime docs](https://github.com/vllm-project/aibrix/blob/main/docs/source/features/runtime.rst#metric-standardization).

## Contributing
We welcome contributions from the community! Check out our [contributing guidelines](https://github.com/vllm-project/aibrix/blob/main/CONTRIBUTING.md) to see how you can make a difference.

### Build from source

```bash
# This may take several minutes
pip install -e .
```

### Lint, Format and Type Check

Before contributing your code, please run the following commands to ensure that your code passes the tests and linting checks.

```bash
# install dependencies
poetry install --no-root --with dev

# linting, formatting and type checking
bash ./scripts/format.sh
```

## License

AI Runtime is licensed under the [APACHE License](https://github.com/vllm-project/aibrix/LICENSE.md).
