We’re excited to share new id and entry administration options to assist simplify the set-up and scale of Databricks for admins. Unity Catalog is on the middle of governance on the Databricks Information Intelligence Platform. A part of Unity Catalog is our id and entry administration capabilities, designed with the next rules:

1. Construct safe, scalable, and ubiquitous id and entry administration for onboarding, managing, and collaborating.
2. Allow clients to simply management entry to Databricks utilizing intuitive, extensible, and audit-ready permissions.
3. Develop world-class, extremely scalable authentication for browser and API entry to allow clients and companions to easily and securely leverage the ability of the Databricks Information Intelligence Platform.

On this weblog, we’ll present a refresher on present id and entry administration options and introduce new investments to simplify the Databricks admin expertise. These investments embrace easy logins from Energy BI and Tableau, simplified single sign-on setup through unified login, OAuth authorization, and working jobs utilizing the id of a service principal as a safety greatest apply.

Seamlessly join Energy BI and Tableau to Databricks on AWS utilizing single sign-on

Energy BI and Tableau are two of the preferred third celebration knowledge instruments on Databricks. The power to securely join from Energy BI and Tableau to Databricks with single sign-on is now usually accessible on AWS. Databricks leverages OAuth to permit customers to entry Databricks from these instruments utilizing single sign-on. This simplifies login for customers and reduces the chance of leaked credentials. OAuth accomplice purposes for Energy BI and Tableau are enabled in your account by default.

To get began, take a look at our docs web page or watch this demo video for Energy BI.

Authenticate customers to Databricks utilizing unified login on AWS

Single sign-on (SSO) is a key safety greatest apply and permits you to authenticate your customers to Databricks utilizing your most popular id supplier. At Databricks, we provide SSO throughout all three clouds. On Azure and GCP, we provide SSO to your account and workspaces by default within the type of Microsoft Entra ID (previously Azure Lively Listing) and Google Cloud Identification, respectively. On AWS, Databricks affords help for a wide range of id suppliers resembling Okta, Microsoft Entra ID, and OneLogin utilizing both SAML or OIDC.

This summer season, we launched unified login, a brand new characteristic that simplifies SSO for Databricks on AWS accounts and workspaces. Unified login permits you to handle one SSO configuration in your account and each Databricks workspace related to it. With Single Signal-On (SSO) activated in your account, you possibly can allow unified login for all or particular workspaces. This setup makes use of an account-level SSO configuration for Databricks entry, simplifying consumer authentication throughout your account’s workspaces. Unified Login is in use on hundreds of workspaces in manufacturing already.

Unified login is GA and enabled robotically on accounts created after June 21, 2023. The characteristic is in public preview for accounts created earlier than June 21, 2023. To allow unified login, see arrange SSO in your Databricks account console.

Automate service principal entry to Databricks with OAuth on AWS

We’re excited to announce that OAuth for service principals is usually accessible on AWS. On Azure and GCP, we help OAuth through Azure and Google tokens, respectively. Service principals are Databricks identities to be used with automated instruments, jobs, and purposes. It’s a safety greatest apply to make use of service principals as an alternative of customers for manufacturing automation workflows for the next causes:

• Manufacturing workflows that run utilizing service principals aren’t impacted when customers depart the group or change roles.
• If all processes that act on manufacturing knowledge run utilizing service principals, interactive customers don’t want any write, delete, or modify privileges in manufacturing. This eliminates the chance of a consumer overwriting manufacturing knowledge by chance.
• Utilizing service principals for automated workflows permits customers to raised defend their very own entry tokens.

OAuth is an open normal protocol that authorizes customers and repair accounts to APIs and different sources with out revealing the credentials. OAuth for service principals makes use of the OAuth consumer credentials movement to generate OAuth entry tokens that can be utilized to authenticate to Databricks APIs. OAuth for service principals has the next advantages for authenticating to Databricks:

• Makes use of Databricks service principals, as an alternative of customers, for authentication.
• Makes use of short-lived (one-hour) entry tokens for credentials to cut back the chance of credentials being leaked.
• Expired OAuth entry token might be robotically refreshed utilizing Databricks instruments and SDKs.
• Can authenticate to all Databricks APIs that the service principal has entry to, at each the account and workspace stage. This lets you automate the creation and setup of workspaces in a single script.

To make use of OAuth for service principals, see Authentication utilizing OAuth for service principals.

Securely run Databricks jobs as a Service Principal

Databricks Workflows orchestrates knowledge processing, machine studying, and analytics pipelines within the Databricks Information Intelligence Platform. A Databricks job is a method to run your knowledge processing and evaluation purposes in a Databricks workspace. By default, jobs run because the id of the job proprietor. Which means that the job assumes the permissions of the job proprietor and might solely entry knowledge and Databricks objects that the job proprietor has permission to entry.

We’re excited to announce that you would be able to now change the id that the job is working as to a service principal. Which means that the job assumes the permissions of that service principal as an alternative of the proprietor and ensures that the job is not going to be affected by a consumer leaving your group or switching departments. Working a job as a service principal is usually accessible on AWS, Azure, and GCP. Try Run a job as a service principal within the docs to get began.

“Working Databricks workflows utilizing service principals permits us to separate the workflows permissions, their execution, and their lifecycle from customers, and subsequently making them safer and strong”

— George Moldovan, Product Proprietor, Raiffeisen Financial institution Worldwide

Identification and Entry Administration Finest Practices on Databricks

At Databricks, we’re dedicated to scaling with you as your group grows. We lined quite a bit in right this moment’s weblog, highlighting our key investments in our id and entry administration platform through Unity Catalog on Databricks. With a slew of recent id and entry administration options now accessible, you may marvel what “good” seems to be like as you construct your knowledge governance technique with Databricks.

We suggest you take a look at our id and entry administration docs pages for the newest greatest practices (AWS | Azure | GCP) or watch our Information + AI Summit 2023 session “Finest Practices for Setting Up Databricks SQL at Enterprise Scale”.