airflow-service

Description

Install Postgresql service and create a database. Then install Airflow and configure it to use that database.

TODO: reverse proxy, other options

Variables

Name Type Default Description

airflow_executor

string LocalExecutor

The executor class that airflow should use.

Choices include:

  • SequentialExecutor
  • LocalExecutor
  • CeleryExecutor
  • DaskExecutor

Code

doc:
  short_help: Setup Airflow service.
  help: |
    Install Postgresql service and create a database. Then install Airflow and configure it to use that database.

    TODO: reverse proxy, other options

args:
  airflow_executor:
#    type: string
    doc: |
      The executor class that airflow should use.

      Choices include:

      - SequentialExecutor
      - LocalExecutor
      - CeleryExecutor
      - DaskExecutor
    type: string
    allowed:
    - SequentialExecutor
    - LocalExecutor
    - CeleryExecutor
    - DaskExecutor
    default: LocalExecutor



frecklets:
- postgresql-database-exists:
    db_name: airflow
    db_user: airflow
    db_user_password: md5053710be12d3c1a99ff746d66b84628c
    postgresql_listen_addresses:
    - localhost
    - 10.0.0.22
#      postgresql_listen_addresses:
#        - localhost
#        - 192.168.178.204
#        - 192.168.194.204
    postgresql_pg_hba:
    - method: md5
    setup_postgresql: true
- frecklet:
    type: ansible-role
    name: idealista.airflow-role
    desc:
      short: install Airflow service
    properties:
      elevated: true
      internet: true
      idempotent: true
  task:
    become: true
    include-type: import
  vars:
    airflow_required_python_packages:
    - name: pyasn1
      version: 0.4.4
#      airflow_webserver_base_url: http://10.0.0.22:8080
    airflow_executor: '{{:: airflow_executor ::}}'
    airflow_load_examples: false
#      airflow_database_conn: "postgresql+psycopg2://airflow:WK+*g2'ltj{[email protected]:5432/airflow"
    airflow_database_conn: postgresql+psycopg2://airflow:WK+*g2'ltj{[email protected]:5432/airflow
frecklecute --community airflow-service --help

Usage: frecklecute airflow-service [OPTIONS]

  Install Postgresql service and create a database. Then install Airflow and
  configure it to use that database.

  TODO: reverse proxy, other options

Options:
  --airflow-executor AIRFLOW_EXECUTOR
                                  The executor class that airflow should use.
  --help                          Show this message and exit.
# -*- coding: utf-8 -*-


#
# module path: pycklets.airflow_service.AirflowService
#


from dataclasses import dataclass
from pyckles import AutoPycklet
from typing import *    # noqa

@dataclass
class AirflowService(AutoPycklet):
    """Install Postgresql service and create a database. Then install Airflow and configure it to use that database.

     TODO: reverse proxy, other options

       Args:
         airflow_executor: The executor class that airflow should use.

    """

    FRECKLET_ID = "airflow-service"

    airflow_executor: str = None


    def __post_init__(self):
        super(AirflowService, self).__init__(var_names=["airflow_executor"])


frecklet_class = AirflowService
# -*- coding: utf-8 -*-


#
# module path: pycklets.airflow_service.AirflowService
#


from pyckles import AutoPycklet

class AirflowService(AutoPycklet):
    """Install Postgresql service and create a database. Then install Airflow and configure it to use that database.

     TODO: reverse proxy, other options

       Args:
         airflow_executor: The executor class that airflow should use.

    """

    FRECKLET_ID = "airflow-service"

    def __init__(self, airflow_executor="LocalExecutor"):

        super(AirflowService, self).__init__(var_names=["airflow_executor"])
        self._airflow_executor = airflow_executor

    @property
    def airflow_executor(self):
        return self._airflow_executor

    @airflow_executor.setter
    def airflow_executor(self, airflow_executor):
        self._airflow_executor = airflow_executor



frecklet_class = AirflowService