Databricks Notebooks affords builders a managed authoring expertise the place knowledge and AI groups can effectively collaborate on tasks collectively. The workforce right here is working exhausting as we put together to share thrilling new improvements for Notebooks on the Information + AI Summit later this month, and we hope you’ll be part of us at our session Develop Like a Professional in Databricks Notebooks, led by Weston Hutchins and Neha Sharma.  As a heat up, we needed to take a fast look again at some latest additions in Notebooks.

Run Databricks Notebooks on SQL Warehouses

SQL is the 2nd hottest language in Notebooks behind Python, and to higher assist our customers who love SQL, we’re bringing SQL warehouses to Notebooks.  SQL warehouses are the identical sources that energy Databricks SQL, they usually ship higher price-performance for SQL execution in comparison with all-purpose clusters.  This characteristic is rolling out now, so preserve a watch out!

Whereas connected to a SQL warehouse, solely your pocket book’s SQL cells will execute. Cells utilizing different languages (like Python or Scala) shall be skipped. Markdown cells will proceed to be rendered. To be taught extra, go to our documentation.

View knowledge from Notebooks, SQL editor, and Information Explorer in the identical expertise

The brand new unified schema browser enables you to view the entire knowledge within the Unity Catalog metastore with out leaving a pocket book or the SQL editor. You’ll be able to choose “For you” to filter the checklist to the lively tables in your pocket book. 

As you kind your search request into the filter field, the show actively updates to point out solely these gadgets that include that textual content. This can search for gadgets which might be at present open or have been opened earlier within the present session. Study extra right here.

Share Notebooks utilizing Delta Sharing

Now you can use Delta Sharing to share pocket book recordsdata with the Databricks-to-Databricks sharing circulation. You get the convenience and safety of Delta Sharing. Sharing notebooks empowers you to collaborate throughout metastores and accounts. This permits individuals who share knowledge to unpack the worth of that knowledge with notebooks.

To be taught extra, right here is how one can add notebooks to a share (for suppliers) and learn shared notebooks (for recipients).

Debug your Notebooks with the Variable Explorer

The Variable Explorer shows the state of all of the Python variables in your pocket book improvement session. The identify, kind, and worth are surfaced for all easy variable sorts. The Variable Explorer additionally surfaces further metadata for Spark and Pandas DataFrames. The form and column names can be found at-a-glance, and a full view of the schema is on the market on hover. 

The Variable Explorer additionally lets you step by way of and debug Python code by leveraging the assist for pdb in Databricks Notebooks. You’ll be able to set breakpoints with breakpoint() or pdb.set_trace(). While you run the cell, the execution will pause on the breakpoint and the Variable Explorer will robotically replace with the state of the pocket book at that breakpoint. See our documentation for extra data.

See you at Summit

At Information + AI Summit 2023, we’ll have deep dive classes into utilizing Notebooks. We may also discuss in regards to the latest methods to be extra environment friendly whereas utilizing Databricks. We hope to see you there.