Metadata-Version: 1.1
Name: airflow-run
Version: 0.0.8
Summary: Simplified Airflow CLI Tool for Lauching CeleryExecutor Deployment
Home-page: https://github.com/paulokuong/airflow-run
Author: Paulo Kuong
Author-email: paulo.kuong@gmail.com
License: MIT
Description: Airflow Run
        ----------------
        
        Python tool for deploying Airflow Multi-Node Cluster.
        
        Requirements
        ------------
        
        -  Python >=3.6 (tested)
        
        Goal
        ----
        
        | To provide a quick way to setup Airflow Multi-Node Cluster (a.k.a. Celery Executor Setup).
        
        Steps
        -----
        1. Generate config yaml file.
        2. Run commands to start Rabbitmq, Postgresql and other Airflow services:
        
        Generate config file:
        ---------------------
        
        .. code:: python
        
            afr --generate_config
        
        Running the tool in the same directory as config file:
        ------------------------------------------------------
        
        .. code:: python
        
            afr --run postgresql
            afr --run initdb
            afr --run rabbitmq
            afr --run webserver
            afr --run scheduler
            afr --run worker --queue {queue name}
            afr --run flower
        
        Or, running the tool specifying config path:
        --------------------------------------------
        
        .. code:: python
        
            afr --run postgresql --config /path/config.yaml
        
        Default Config yaml file:
        -------------------------
        
        .. code-block:: yaml
        
            private_registry: False
            registry_url: registry.hub.docker.com
            username: ""
            password: ""
            repository: pkuong/airflow-run
            image: airflow-run
            tag: latest
            local_dir: {local directory where you want to mount /dags and /logs folder}
            webserver_port: 8000
            flower_port: 5555
            airflow_cfg:
              AIRFLOW__CORE__EXECUTOR: CeleryExecutor
              AIRFLOW__CORE__LOAD_EXAMPLES: "False"
              AIRFLOW__CORE__DAGS_FOLDER: /usr/local/airflow/airflow/dags
              AIRFLOW__CORE__LOGS_FOLDER: /usr/local/airflow/airflow/logs
              AIRFLOW_HOME: /usr/local/airflow
            rabbitmq:
              name: rabbitmq
              username: {username}
              password: {password}
              host: {IP}
              virtual_host: /
              image: rabbitmq:3-management
              home: /var/lib/rabbitmq
              ui_port: 15672
              port: 5672
              env:
                RABBITMQ_DEFAULT_USER: {username}
                RABBITMQ_DEFAULT_PASS: {password}
            postgresql:
              name: postgresql
              username: {username}
              password: {password}
              host: {host}
              image: postgres
              data: /var/lib/postgresql/data
              port: 5432
              env:
                PGDATA: /var/lib/postgresql/data/pgdata
                POSTGRES_USER: {username}
                POSTGRES_PASSWORD: {password}
        
        
        Docker image
        ------------
        
        | This tool is using the following public docker image by default.
        
        .. code:: python
        
            https://hub.docker.com/repository/docker/pkuong/airflow-run
        
        Building the image:
        -------------------
        
        | If you want to build your own image, you can run the following:
        
        .. code:: python
        
            afd --build --config_path={absolute path to config.yaml} --dockerfile_path={absolute path to directory which contains Dockerfile}
        
        Contributors
        ------------
        
        -  Paulo Kuong (`@pkuong`_)
        
        .. _@pkuong: https://github.com/paulokuong
        
        .. |Build Status| image:: https://travis-ci.org/paulokuong/airflow-run.svg?branch=master
        .. target: https://travis-ci.org/paulokuong/airflow-run
        
Keywords: Airflow CeleryExecutor distributed portable deployment runner docker kubernetes
Platform: UNKNOWN
Classifier: Development Status :: 3 - Alpha
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.4
