Lucas Fonseca
Jul 2, 2021

Thank you, Jéssica. Currently, our platform is thought considering Airflow, but it's a good observation! We do consider leveraging this solution to other orchestrators, since this is a generic strategy; so we may explore adding it to Jenkins, Rundeck, etc.

Please take a look in Databricks post: Integrating Apache Airflow with Databricks and in the Airflow documenation about DatabricksSubmitRunOperator.

Lucas Fonseca
Lucas Fonseca

Written by Lucas Fonseca

Data Engineering Lead @Inventa. Data lover, curious spirit, amateur musician, and Gym rat.

No responses yet