Metadata-Version: 2.1
Name: spetlr
Version: 2.0.6
Summary: A python ETL libRary (SPETLR) for Databricks powered by Apache SPark.
Home-page: https://github.com/spetlr-org/spetlr
Author: Spetlr.Org
Author-email: spetlr.org@gmail.com
License: UNKNOWN
Project-URL: Documentation, https://github.com/spetlr-org/spetlr
Project-URL: Bug Reports, https://github.com/spetlr-org/spetlr/issues
Project-URL: Source Code, https://github.com/spetlr-org/spetlr
Description: # spetlr
        
        A python ETL libRary (SPETLR) for Databricks powered by Apache SPark.
        
        Visit SPETLR official webpage: [https://spetlr.com/](https://spetlr.com/)
        
        # Table of Contents
        
        - [Description](#description)
        - [Important Notes](#important-notes)
        - [Installation](#installation)
        - [Development Notes](#development-notes)
        - [Testing](#testing)
        - [General Project Info](#general-project-info)
        - [Contributing](#contributing)
        - [Build Status](#build-status)
        - [Releases](#releases)
        - [Contact](#contact)
        
        # Description
        
        SPETLR has a lot of great tools for working with ETL in Databricks. But to make it easy for you to consider why you need
        SPETLR here is a list of the core features:
        
        * ETL framework: A common ETL framework that enables reusable transformations in an object-oriented manner. Standardized
          structures facilitate cooperation in large teams.
        
        * Integration testing: A framework for creating test databases and tables before deploying to production in order to
          ensure reliable and stable data platforms. An additional layer of data abstraction allows full integration testing.
        
        * Handlers: Standard connectors with commonly used options reduce boilerplate.
        
        For more information, visit SPETLR official webpage: [https://spetlr.com/](https://spetlr.com/)
        
        # Important Notes
        
        This package can not be run or tested without access to `pyspark`.
        However, installing `pyspark` as part of our installer gave issues when
        other versions of `pyspark` were needed. Hence we took out the dependency
        from our installer.
        
        # Installation
        
        Install SPETLR from PyPI:
        [![PyPI version](https://badge.fury.io/py/spetlr.svg)](https://pypi.org/project/spetlr/)
        [![PyPI](https://img.shields.io/pypi/dm/spetlr)](https://pypi.org/project/spetlr/)
        
        ```    
        pip install spetlr
        ```
        
        # Development Notes
        
        To prepare for development, please install these additional requirements:
        
        - Java 8
        - `pip install -r test_requirements.txt`
        
        Then install the package locally
        
            python setup.py develop
        
        ## Testing
        
        ### Local tests
        
        After installing the dev-requirements, execute tests by running:
        
            pytest tests
        
        These tests are located in the `./tests/local` folder and only require a Python interpreter. Pull requests will not be
        accepted if these tests do not pass. If you add new features, please include corresponding tests.
        
        ### Cluster tests
        
        Tests in the `./tests/cluster` folder are designed to run on a Databricks cluster.
        The [Pre-integration Test](https://github.com/spetlr-org/spetlr/blob/main/.github/workflows/pre-integration.yml)
        utilizes Azure Resource deployment - and can only be run by the spetlr-org admins.
        
        To deploy the necessary Azure resources to your own Azure Tenant, run the following command:
        
        ```powershell
        .\.github\deploy\deploy.ps1 -uniqueRunId "yourUniqueId"
        ```
        
        Afterward, execute the following commands:
        
        ```powershell 
        .\.github\submit\build.ps1
        .\.github\submit\submit_test_job.ps1
        ```
        
        # General Project Info
        
        [![Github top language](https://img.shields.io/github/languages/top/spetlr-org/spetlr)](https://github.com/spetlr-org/spetlr)
        [![Github stars](https://img.shields.io/github/stars/spetlr-org/spetlr)](https://github.com/spetlr-org/spetlr)
        [![Github forks](https://img.shields.io/github/forks/spetlr-org/spetlr)](https://github.com/spetlr-org/spetlr)
        [![Github size](https://img.shields.io/github/repo-size/spetlr-org/spetlr)](https://github.com/spetlr-org/spetlr)
        [![Issues Open](https://img.shields.io/github/issues/spetlr-org/spetlr.svg?logo=github)](https://github.com/spetlr-org/spetlr/issues)
        [![PyPI spetlr badge](https://img.shields.io/pypi/v/spetlr)](https://pypi.org/project/spetlr/)
        
        # Contributing
        
        Feel free to contribute to SPETLR. Any contributions are appreciated - not only new features, but also if you find a way
        to improve SPETLR.
        
        If you have a suggestion that can enhance SPETLR, please fork the repository and create a pull request. Alternatively,
        you can open an issue with the "enhancement" tag.
        
        1. Fork the Project
        2. Create your Feature Branch (`git checkout -b feature/NewSPETLRFeature`)
        3. Commit your Changes (`git commit -m 'Add some SEPTLRFeature'`)
        4. Push to the Branch (`git push origin feature/NewSPETLRFeature`)
        5. Open a Pull Request
        
        # Build Status
        
        [![Post-Integration](https://github.com/spetlr-org/spetlr/actions/workflows/post-integration.yml/badge.svg)](https://github.com/spetlr-org/spetlr/actions/workflows/post-integration.yml)
        
        # Releases
        
        Releases to PyPI is an Github Action which needs to be manually triggered.
        
        [![Release](https://github.com/spetlr-org/spetlr/actions/workflows/release.yml/badge.svg)](https://github.com/spetlr-org/spetlr/actions/workflows/release.yml)
        [![PyPI spetlr badge](https://img.shields.io/pypi/v/spetlr)](https://pypi.org/project/spetlr/)
        
        # Contact
        
        For any inquiries, please use the [SPETLR Discord Server](https://discord.gg/p9bzqGybVW).
        
Keywords: databricks,pyspark
Platform: UNKNOWN
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Build Tools
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.8
Description-Content-Type: text/markdown
