Load the image into kind: kind load docker-image my-dags:0.0.1 Then build the image: docker build -t my-dags:0.0.1. mkdir my-airflow-project & cd my-airflow-project You can test locally before pushing to kind with astro airflow start. Start a project using astro-cli, which will generate a Dockerfile, and load your DAGs in.
To port-forward the Airflow UI to to confirm Airflow is working. Run kubectl port-forward svc/airflow-webserver 8080:8080 -n airflow Confirm the pods are up: kubectl get pods -all-namespaces
Helm install airflow -n airflow astronomer/airflow We recommend testing with Kubernetes 1.16+, example: kind create cluster \Ĭonfirm it's up: kubectl cluster-info -context kind-kindĪdd Astronomer's Helm repo: helm repo add astronomer Ĭreate namespace + install the chart: kubectl create namespace airflow (Note: KEDA does not support StatefulSets so you need to set to false) helm repo add astronomer Īstronomer/airflow Walkthrough using kind On this chart by setting =true your helm command or in the values.yaml. Once KEDA is installed (which should be pretty quick since there is only one pod). set tricsAdapter=docker.io/kedacore/keda-metrics-adapter:1.2.0 \ set image.keda=docker.io/kedacore/keda:1.2.0 \ To install our custom version of KEDA on your cluster, please run helm repo add kedacore On a separate branch, but will be merged upstream soon. We've built an experimental scaler that allows users to create scalers based on postgreSQL queries. To the Kubernetes Horizontal Pod Autoscaler. KEDA is a custom controller that allows users to create custom bindings KEDA stands for Kubernetes Event Driven Autoscaling. For example, helm install -name my-release \ Specify each parameter using the -set key=value argument to helm install. The K8s pullPolicy for the the auth sidecar proxy image More about Extra Objects.Įnable security context constraints required for OpenShift Įxtra K8s Objects to deploy (these are passed through tpl). Name of secret that contains a TLS secretĪnnotations added to Webserver Ingress objectĪnnotations added to Flower Ingress object The following tables lists the configurable parameters of the Astronomer chart and their default values. The complete list of parameters supported by the community chart can be found on the Parameteres Reference page, and can be set under the airflow key in this chart. Other non-airflow images used in this chart are generated from this repository.The Airflow image that are referenced as the default values in this chart are generated from this repository.Value: /usr/local/airflow/dags/latest/airflow/example_dags The following example relies on emptyDir volumes and works with KubernetesExecutor. set =8a0da78 Deploying DAGs using git-syncĮxtraContainers, extraInitContainers, extraVolumes, and extraVolumeMounts can be combined to deploy git-sync.
Registry ( docker push my-company/airflow:8a0da78), then update the Airflow pods with that image: helm upgrade my-release.
Latest code ( docker build -t my-company/airflow:8a0da78.
The recommended way to update your DAGs with this chart is to build a new docker image with the The command removes all the Kubernetes components associated with the chart and deletes the release. To uninstall/delete the my-release deployment: helm delete my-release To upgrade the chart with the release name my-release: helm upgrade -name my-release. Tip: List all releases using helm list Upgrading the Chartįirst, look at the updating documentation to identify any backwards-incompatible changes. The Parameters section lists the parameters that can be configured during installation. The command deploys Airflow on the Kubernetes cluster in the default configuration. To install the chart with the release name my-release: helm install -name my-release.
Astronomer's Helm Chart for Apache Airflow