Metadata-Version: 2.4
Name: aioscraper
Version: 0.2.1
Summary: Async framework for building modular and scalable web scrapers.
Author: darkstussy
Project-URL: Homepage, https://github.com/darkstussy/aioscraper
Project-URL: Issues, https://github.com/darkstussy/aioscraper/issues
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Framework :: AsyncIO
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Topic :: Internet :: WWW/HTTP :: Indexing/Search
Classifier: Topic :: Software Development :: Libraries :: Application Frameworks
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: aiohttp[speedups]~=3.12.15
Requires-Dist: aiojobs~=1.4.0
Provides-Extra: dev
Requires-Dist: flake8~=7.3.0; extra == "dev"
Requires-Dist: black~=25.1.0; extra == "dev"
Requires-Dist: pyright~=1.1.404; extra == "dev"
Requires-Dist: aiohttp[speedups]~=3.12.15; extra == "dev"
Requires-Dist: aiojobs~=1.4.0; extra == "dev"
Provides-Extra: test
Requires-Dist: pytest~=8.4.1; extra == "test"
Requires-Dist: pytest-asyncio~=1.1.0; extra == "test"
Requires-Dist: aresponses~=3.0.0; extra == "test"
Dynamic: license-file

# aioscraper

**Asynchronous framework for building modular and scalable web scrapers.**

![Python](https://img.shields.io/badge/python-3.10%2B-blue)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
![Version](https://img.shields.io/github/v/tag/darkstussy/aioscraper?label=version)

## Features

- 🚀 Fully asynchronous architecture powered by `aiohttp` and `aiojobs`
- 🔧 Modular system with middleware support
- 📦 Pipeline data processing
- ⚙️ Flexible configuration
- 🔄 Priority-based request queue management
- 🛡️ Built-in error handling

## Installation

```bash
pip install aioscraper
```

## Requirements

- Python 3.10 or higher
- aiohttp
- aiojobs

## Quick Start

```python
import asyncio

from aioscraper import BaseScraper, AIOScraper
from aioscraper.types import Response, RequestSender


class Scraper(BaseScraper):
    async def start(self, send_request: RequestSender) -> None:
        await send_request(url="https://example.com", callback=self.parse)

    async def parse(self, response: Response) -> None:
        # handle response
        pass


async def main():
    async with AIOScraper(scrapers=[Scraper()]) as scraper:
        await scraper.start()


if __name__ == "__main__":
    asyncio.run(main())
```

## License

MIT License

Copyright (c) 2025 darkstussy

## Links

- [PyPI](https://pypi.org/project/aioscraper)
- [GitHub](https://github.com/darkstussy/aioscraper)
- [Issues](https://github.com/darkstussy/aioscraper/issues)
