Hi, glad you enjoyed it. We actually customized the official Databricks Operator (for Airflow). The behavior is being kept, with a few tweaks like variable encapsulation on the operator's constructor, in order to make it cleaner to end-users =].

Also, answering your other question: we create a different (and dedicated) cluster for each DAG. So, currently, there is no re-utilization of clusters.