Databricks Notebooks gives builders a managed authoring expertise the place information and AI groups can effectively collaborate on initiatives collectively. The staff right here is working onerous as we put together to share thrilling new improvements for Notebooks on the Information + AI Summit later this month, and we hope you’ll be part of us at our session Develop Like a Professional in Databricks Notebooks, led by Weston Hutchins and Neha Sharma.  As a heat up, we needed to take a fast look again at some current additions in Notebooks.

Run Databricks Notebooks on SQL Warehouses

SQL is the 2nd hottest language in Notebooks behind Python, and to higher help our customers who love SQL, we’re bringing SQL warehouses to Notebooks.  SQL warehouses are the identical assets that energy Databricks SQL, and so they ship higher price-performance for SQL execution in comparison with all-purpose clusters.  This function is rolling out now, so it is possible for you to to connect your notebooks to a SQL warehouse very quickly!

Whereas connected to a SQL warehouse, solely your pocket book’s SQL cells will execute. Cells utilizing different languages (like Python or Scala) will probably be skipped. Markdown cells will proceed to be rendered. To be taught extra, go to our documentation.

View information from Notebooks, SQL editor, and Information Explorer in the identical expertise

The brand new unified schema browser helps you to view the entire information within the Unity Catalog metastore with out leaving a pocket book or the SQL editor. You possibly can choose “For you” to filter the checklist to the energetic tables in your pocket book. 

As you sort your search request into the filter field, the show actively updates to point out solely these gadgets that comprise that textual content. This can search for gadgets which are at the moment open or have been opened earlier within the present session. Study extra right here.

Share Notebooks utilizing Delta Sharing

Now you can use Delta Sharing to share pocket book recordsdata with the Databricks-to-Databricks sharing movement. You get the convenience and safety of Delta Sharing. Sharing notebooks empowers you to collaborate throughout metastores and accounts. This permits individuals who share information to unpack the worth of that information with notebooks.

To be taught extra, right here is how one can add notebooks to a share (for suppliers) and find out how to learn shared notebooks (for recipients).

Debug your Notebooks with the Variable Explorer

The Variable Explorer shows the state of all of the Python variables in your pocket book growth session. The identify, sort, and worth are surfaced for all easy variable sorts. The Variable Explorer additionally surfaces further metadata for Spark and Pandas DataFrames. The form and column names can be found at-a-glance, and a full view of the schema is offered on hover. 

The Variable Explorer additionally permits you to step by way of and debug Python code by leveraging the help for pdb in Databricks Notebooks. You possibly can set breakpoints with breakpoint() or pdb.set_trace(). Whenever you run the cell, the execution will pause on the breakpoint and the Variable Explorer will mechanically replace with the state of the pocket book at that breakpoint. See our documentation for extra data.

See you at Summit

At Information + AI Summit 2023, we’ll have deep dive classes into utilizing Notebooks. We will even discuss concerning the latest methods to be extra environment friendly whereas utilizing Databricks. We hope to see you there.