Metadata-Version: 2.4
Name: min-obproject
Version: 0.1.0
Summary: Reusable Metaflow flow templates and configurations
Requires-Python: <3.13,>=3.8
Requires-Dist: ob-project-utils>=0.1.45
Requires-Dist: outerbounds>=0.10.20
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == 'dev'
Requires-Dist: ruff>=0.1.0; extra == 'dev'
Description-Content-Type: text/markdown

# min-obproject

This obproject demonstrates
1. Configuration patterns for machine learning workflows where there are both many parameters inducing a need for a config file, and you want to dynamically override config values.
2. How to build Metaflow flow templates.

## Repository Structure

```
min-obproject/
├── src/                          # Shared Python modules
│   └── flow_templates.py         # Reusable flow base classes
├── flows/                        # Individual Metaflow workflows
│   ├── 01-config-override/       # Example: basic config override pattern
│   └── 02-projectflow-inheritance/  # Example: template inheritance
└── obproject.toml                # Outerbounds project configuration
```

## Key Features

- Makes use of [Metaflow Configs](https://docs.metaflow.org/metaflow/configuring-flows/introduction). 
- **`ConfigurableNeuralNetworkFlow`**: Base class example that auto-resolves a `config.json` assumed to be in an inheriting flow's directory. This pattern helps downstream flows leverage configs for automating experiments with many parameters, while avoiding a lot of boilerplate in FlowSpec-inheriting classes.
- **Dynamic Config Resolution**: Configs can be selectively overridden using individual [Metaflow Parameters](https://docs.metaflow.org/api/flowspec#parameters). This way, we get the best of both worlds, and individual parameters of the config files can be overridden with arbitrarily complex logic that library authors write once, and downstream flow consumers can operate.

## Development Setup

### Install as Editable Package (Local Development)

```bash
# Install uv if needed
curl -LsSf https://astral.sh/uv/install.sh | sh

# Install package in editable mode
uv pip install -e .

# With dev dependencies
uv pip install -e ".[dev]"
```

Now `src` modules are importable from anywhere:
```python
from src.flow_templates import NeuralNetworkFlow
```

### Run Flows

Always run from project root:
```bash
python flows/02-projectflow-inheritance/flow.py run
python flows/02-projectflow-inheritance/flow.py run --with kubernetes
```

## Publishing to PyPI

### Build and Publish

```bash
# Build distribution
uv build

# Publish to PyPI
uv publish

# Or publish to test PyPI first
uv publish --index-url https://test.pypi.org/legacy/
```

### Use Published Package in Production

```python
from metaflow import FlowSpec, step, pypi_base

@pypi_base(packages={"min-obproject": "0.1.0"})
class MyFlow(FlowSpec):
    ...
```

## Example Usage

```python
from metaflow import step
from obproject import ProjectFlow
from src.flow_templates import ConfigurableNeuralNetworkFlow

class MyTrainingFlow(ProjectFlow, ConfigurableNeuralNetworkFlow):
    
    @step
    def start(self):
        self._resolve_config()  # Loads config.json from flow directory
        print(f"Training with lr={self.config['lr']}")
        self.next(self.end)
    
    @step
    def end(self):
        pass
```

Create `config.json` in the same directory as your flow:
```json
{
    "train_args": {
        "lr": 0.001,
        "optimizer": "adam"
    }
}
```

Override config values via CLI:
```bash
python flow.py run --lr 0.01
```

