Metadata-Version: 2.3
Name: ingestr
Version: 0.7.3
Summary: ingestr is a command-line application that ingests data from various sources and stores them in any database.
Project-URL: Homepage, https://github.com/bruin-data/ingestr
Project-URL: Issues, https://github.com/bruin-data/ingestr/issues
Author-email: Burak Karakan <burak.karakan@getbruin.com>
License-File: LICENSE.md
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Database
Requires-Python: >=3.9
Requires-Dist: connectorx==0.3.3
Requires-Dist: cx-oracle==8.3.0
Requires-Dist: databricks-sql-connector==2.9.3
Requires-Dist: dlt==0.5.1
Requires-Dist: duckdb-engine==0.11.5
Requires-Dist: duckdb==0.10.2
Requires-Dist: google-api-python-client==2.130.0
Requires-Dist: google-cloud-bigquery-storage==2.24.0
Requires-Dist: mysql-connector-python==9.0.0
Requires-Dist: pendulum==3.0.0
Requires-Dist: psycopg2-binary==2.9.9
Requires-Dist: py-machineid==0.5.1
Requires-Dist: pymongo==4.6.3
Requires-Dist: pymysql==1.1.0
Requires-Dist: pyodbc==5.1.0
Requires-Dist: pyrate-limiter==3.6.1
Requires-Dist: redshift-connector==2.1.0
Requires-Dist: rich==13.7.1
Requires-Dist: rudder-sdk-python==2.1.0
Requires-Dist: snowflake-sqlalchemy==1.5.3
Requires-Dist: sqlalchemy-bigquery==1.11.0
Requires-Dist: sqlalchemy-hana==2.0.0
Requires-Dist: sqlalchemy-redshift==0.8.14
Requires-Dist: sqlalchemy2-stubs==0.0.2a38
Requires-Dist: sqlalchemy==1.4.52
Requires-Dist: tqdm==4.66.2
Requires-Dist: typer==0.12.3
Description-Content-Type: text/markdown

<div align="center">
    <img src="https://github.com/bruin-data/ingestr/blob/main/resources/ingestr.svg?raw=true" width="500" />
    <p>Ingest & copy data from any source to any destination without any code</p>
    <img src="https://github.com/bruin-data/ingestr/blob/main/resources/demo.gif?raw=true" width="750" />
</div>

<div align="center" style="margin-top: 24px;">
  <a target="_blank" href="https://join.slack.com/t/bruindatacommunity/shared_invite/zt-2dl2i8foy-bVsuMUauHeN9M2laVm3ZVg" style="background:none">
    <img src="https://img.shields.io/badge/slack-join-dlt.svg?color=d95f5f&logo=slack" style="width: 180px;"  />
  </a>
</div>

-----

Ingestr is a command-line application that allows you to ingest data from any source into any destination using simple command-line flags, no code necessary.

- ✨ copy data from your database into any destination
- ➕ incremental loading: `append`, `merge` or `delete+insert`
- 🐍 single-command installation

ingestr takes away the complexity of managing any backend or writing any code for ingesting data, simply run the command and watch the data land on its destination.


## Installation
```
pip install ingestr
```

## Quickstart

```bash
ingestr ingest \
    --source-uri 'postgresql://admin:admin@localhost:8837/web?sslmode=disable' \
    --source-table 'public.some_data' \
    --dest-uri 'bigquery://<your-project-name>?credentials_path=/path/to/service/account.json' \
    --dest-table 'ingestr.some_data'
```

That's it.

This command will:
- get the table `public.some_data` from the Postgres instance.
- upload this data to your BigQuery warehouse under the schema `ingestr` and table `some_data`.

## Documentation
You can see the full documentation [here](https://bruin-data.github.io/ingestr/getting-started/quickstart.html).

## Community
Join our Slack community [here](https://join.slack.com/t/bruindatacommunity/shared_invite/zt-2dl2i8foy-bVsuMUauHeN9M2laVm3ZVg).


## Supported Sources & Destinations

<table>
    <tr>
        <th></th>
        <th>Source</th>
        <th>Destination</th>
    </tr>
    <tr>
        <td colspan="3" style='text-align:center;'><strong>Databases</strong></td>
    </tr>
    <tr>
        <td>Postgres</td>
        <td>✅</td>
        <td>✅</td>
    </tr>
    <tr>
        <td>BigQuery</td>
        <td>✅</td>
        <td>✅</td>
    </tr>
    <tr>
        <td>Snowflake</td>
        <td>✅</td>
        <td>✅</td>
    </tr>
    <tr>
        <td>Redshift</td>
        <td>✅</td>
        <td>✅</td>
    </tr>
    <tr>
        <td>Databricks</td>
        <td>✅</td>
        <td>✅</td>
    </tr>
    <tr>
        <td>DuckDB</td>
        <td>✅</td>
        <td>✅</td>
    </tr>
    <tr>
        <td>Microsoft SQL Server</td>
        <td>✅</td>
        <td>✅</td>
    </tr>
    <tr>
        <td>Local CSV file</td>
        <td>✅</td>
        <td>✅</td>
    </tr>
    <tr>
        <td>MongoDB</td>
        <td>✅</td>
        <td>❌</td>
    </tr>
    <tr>
        <td>Oracle</td>
        <td>✅</td>
        <td>❌</td>
    </tr>
    <tr>
        <td>SAP Hana</td>
        <td>✅</td>
        <td>❌</td>
    </tr>
    <tr>
        <td>SQLite</td>
        <td>✅</td>
        <td>❌</td>
    </tr>
    <tr>
        <td>MySQL</td>
        <td>✅</td>
        <td>❌</td>
    </tr>
    <tr>
        <td colspan="3" style='text-align:center;'><strong>Platforms</strong></td>
    </tr>
    <tr>
        <td>Gorgias</td>
        <td>✅</td>
        <td>❌</td>
    </tr>
    <tr>
        <td>Google Sheets</td>
        <td>✅</td>
        <td>❌</td>
    </tr>
    <tr>
        <td>Notion</td>
        <td>✅</td>
        <td>❌</td>
    </tr>
    <tr>
        <td>Shopify</td>
        <td>✅</td>
        <td>❌</td>
    </tr>
</table>

More to come soon!

## Acknowledgements
This project would not have been possible without the amazing work done by the [SQLAlchemy](https://www.sqlalchemy.org/) and [dlt](https://dlthub.com/) teams. We relied on their work to connect to various sources and destinations, and built `ingestr` as a simple, opinionated wrapper around their work.