Metadata-Version: 2.2
Name: locallab
Version: 0.4.17
Summary: LocalLab: Run language models locally or in Google Collab with a friendly API
Home-page: https://github.com/Developer-Utkarsh/LocalLab
Author: Utkarsh Tiwari
Author-email: utkarshweb2023@gmail.com
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: fastapi<1.0.0,>=0.95.0
Requires-Dist: uvicorn<1.0.0,>=0.21.1
Requires-Dist: pydantic<3.0.0,>=2.0.0
Requires-Dist: python-dotenv<1.0.0,>=0.21.0
Requires-Dist: python-multipart>=0.0.5
Requires-Dist: dataclasses-json<1.0.0,>=0.5.7
Requires-Dist: torch<3.0.0,>=2.0.0
Requires-Dist: transformers<5.0.0,>=4.28.1
Requires-Dist: accelerate<1.0.0,>=0.18.0
Requires-Dist: bitsandbytes<1.0.0,>=0.38.0
Requires-Dist: llama-cpp-python<1.0.0,>=0.1.74
Requires-Dist: click<9.0.0,>=8.1.3
Requires-Dist: rich<14.0.0,>=13.3.4
Requires-Dist: pyngrok<7.0.0,>=6.0.0
Requires-Dist: requests<3.0.0,>=2.28.2
Requires-Dist: netifaces>=0.11.0
Requires-Dist: httpx>=0.24.0
Requires-Dist: colorama>=0.4.4
Requires-Dist: websockets>=10.0
Requires-Dist: psutil>=5.8.0
Requires-Dist: nest-asyncio>=1.5.1
Requires-Dist: fastapi-cache2>=0.2.1
Requires-Dist: nvidia-ml-py3>=7.352.0
Requires-Dist: huggingface_hub>=0.16.0
Requires-Dist: pynvml>=11.0.0
Requires-Dist: typing_extensions>=4.0.0
Requires-Dist: questionary>=1.10.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.15.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: provides-extra
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# 🚀 LocalLab: Run AI Models Easily

[![Build Status](https://img.shields.io/github/actions/workflow/status/Developer-Utkarsh/LocalLab/publish.yml?style=flat-square)](https://github.com/Developer-Utkarsh/LocalLab/actions)
[![LocalLab Version](https://img.shields.io/pypi/v/locallab.svg?style=flat-square)](https://pypi.org/project/locallab/)
[![Python Version](https://img.shields.io/pypi/pyversions/locallab.svg?style=flat-square)](https://pypi.org/project/locallab/)
[![License](https://img.shields.io/github/license/Developer-Utkarsh/LocalLab.svg?style=flat-square)](https://github.com/Developer-Utkarsh/LocalLab/blob/main/LICENSE)

LocalLab empowers users to run any Hugging Face AI model locally or on Google Colab with minimal setup required. It automatically configures an API using ngrok, enabling seamless integration into applications from any location. Designed for simplicity, LocalLab makes advanced AI accessible to all, regardless of technical expertise. With built-in model management, performance optimizations, and system monitoring, it ensures efficient and reliable AI operations for developers, researchers, and enthusiasts alike.

> **Note**: For the latest changes and updates, please see the [CHANGELOG.md](./CHANGELOG.md) file.

## What Problem Does LocalLab Solve?

- **Local Inference:** Run advanced language models without relying on expensive cloud services.
- **Optimized Performance:** Utilize state-of-the-art techniques like quantization, attention slicing, and CPU offloading for maximum efficiency.
- **Seamless Deployment:** Easily switch between local deployment and Google Colab, leveraging ngrok for public accessibility.
- **Effective Resource Management:** Automatically monitor and manage CPU, RAM, and GPU usage to ensure smooth operation.

## System Requirements

### Minimum Requirements

| Component | Local Deployment | Google Colab           |
| --------- | ---------------- | ---------------------- |
| RAM       | 4GB              | Free tier (12GB)       |
| CPU       | 2 cores          | 2 cores                |
| Python    | 3.8+             | 3.8+                   |
| Storage   | 2GB free         | -                      |
| GPU       | Optional         | Available in free tier |

### Recommended Requirements

| Component | Local Deployment | Google Colab       |
| --------- | ---------------- | ------------------ |
| RAM       | 8GB+             | Pro tier (24GB)    |
| CPU       | 4+ cores         | Pro tier (4 cores) |
| Python    | 3.9+             | 3.9+               |
| Storage   | 5GB+ free        | -                  |
| GPU       | CUDA-compatible  | Pro tier GPU       |

## Key Features

- **Interactive CLI:** Configure and run your server with an intuitive command-line interface that adapts to your environment.
- **Multiple Model Support:** Pre-configured models along with the ability to load custom ones on demand.
- **Advanced Optimizations:** Support for FP16, INT8, and INT4 quantization, Flash Attention, and attention slicing.
- **Comprehensive Logging System:** Colorized console output with server status tracking, request monitoring, and performance metrics.
- **Robust Resource Monitoring:** Real-time insights into system performance and resource usage.
- **Flexible Client Libraries:** Comprehensive clients available for both Python and Node.js.
- **Google Colab Friendly:** Dedicated workflow for deploying via Google Colab with public URL access.
- **Persistent Configuration:** Save your settings for future use with the new configuration system.

## Unique Visual Overview

Below is a high-level diagram of LocalLab's architecture.

```mermaid
graph TD
    A["User"] --> B["LocalLab Client (Python/Node.js)"]
    B --> C["LocalLab Server"]
    C --> D["Model Manager"]
    D --> E["Hugging Face Models"]
    C --> F["Optimizations"]
    C --> G["Resource Monitoring"]
```

## Google Colab Workflow

```mermaid
sequenceDiagram
    participant U as "User (Colab)"
    participant S as "LocalLab Server"
    participant N as "Ngrok Tunnel"
    U->>S: Run start_server(ngrok=True)
    S->>N: Establish public tunnel
    N->>U: Return public URL
    U->>S: Connect via public URL
```

## Documentation & Usage Guides

For full documentation and detailed guides, please visit our [documentation page](https://github.com/Developer-Utkarsh/LocalLab/blob/main/docs/README.md).

- [Getting Started Guide](https://github.com/Developer-Utkarsh/LocalLab/blob/main/docs/guides/getting-started.md)
- [CLI Guide](https://github.com/Developer-Utkarsh/LocalLab/blob/main/docs/guides/cli.md)
- [Python Client](https://github.com/Developer-Utkarsh/LocalLab/blob/main/docs/clients/python/README.md)
- [Node.js Client](https://github.com/Developer-Utkarsh/LocalLab/blob/main/docs/clients/nodejs/README.md)
- [Client Comparison](https://github.com/Developer-Utkarsh/LocalLab/blob/main/docs/clients/comparison.md)
- [Google Colab Guide](https://github.com/Developer-Utkarsh/LocalLab/blob/main/docs/colab/README.md)
- [API Reference](https://github.com/Developer-Utkarsh/LocalLab/blob/main/docs/guides/api.md)

## Get Started

1. **Installation:**

   ```bash
   pip install locallab
   ```

2. **Using the CLI (New!):**

   ```bash
   # Start the server with interactive configuration
   locallab start

   # Start with specific options
   locallab start --model microsoft/phi-2 --quantize --quantize-type int8

   # Run the configuration wizard without starting the server
   locallab config

   # Display system information
   locallab info
   ```

3. **Starting the Server Programmatically:**

   ```python
   from locallab import start_server
   start_server()
   ```

4. **Starting the Server on Google Colab:**

   ```python
   !pip install locallab

   # Set up your ngrok auth token (REQUIRED for public access)
   # Get your free token from: https://dashboard.ngrok.com/get-started/your-authtoken
   import os
   os.environ["NGROK_AUTH_TOKEN"] = "your_token_here"

   # Optional: Configure model and optimizations
   os.environ["HUGGINGFACE_MODEL"] = "microsoft/phi-2"  # Choose your preferred model
   os.environ["LOCALLAB_ENABLE_QUANTIZATION"] = "true"  # Enable model optimizations

   # Start the server with ngrok for public access
   from locallab import start_server
   start_server(use_ngrok=True)  # Creates a public URL accessible from anywhere
   ```

5. **Connecting your Client:**

   ```python
   from locallab.client import LocalLabClient

   # Use the ngrok URL displayed in the output above
   client = LocalLabClient("https://xxxx-xxx-xxx-xxx.ngrok.io")

   # Test the connection
   response = client.generate("Hello, how are you?")
   print(response)
   ```

## CLI Features (New in v0.4.8!)

LocalLab now includes a powerful command-line interface with the following features:

- **Interactive Configuration:** Guided setup for all server settings
- **Environment Detection:** Smart defaults based on your system
- **Persistent Settings:** Configuration stored in `~/.locallab/config.json`
- **System Information:** Detailed insights about your hardware
- **Performance Optimizations:** Easy configuration of quantization and other optimizations
- **Google Colab Integration:** Automatic detection and configuration for Colab environments

Example CLI usage:

```bash
# Start with interactive prompts
locallab start

# Configure with specific options
locallab start --model microsoft/phi-2 --port 8080 --quantize --attention-slicing

# Run configuration wizard
locallab config

# Check system resources
locallab info
```

## Join the Community

- Report issues on our [GitHub Issues](https://github.com/Developer-Utkarsh/LocalLab/issues).
- Participate in discussions on our [Community Forum](https://github.com/Developer-Utkarsh/LocalLab/discussions).
- Learn how to contribute by reading our [Contributing Guidelines](https://github.com/Developer-Utkarsh/LocalLab/blob/main/docs/guides/contributing.md).

---

LocalLab is designed to bring the power of advanced language models directly to your workspace—efficiently, flexibly, and affordably. Give it a try and revolutionize your AI projects!
