Driving Self-service and Enhancing DataOps with Atlan

The Lively Metadata Pioneers sequence options Atlan prospects who’ve not too long ago accomplished a radical analysis of the Lively Metadata Administration market. Paying ahead what you’ve realized to the subsequent knowledge chief is the true spirit of the Atlan group! In order that they’re right here to share their hard-earned perspective on an evolving market, what makes up their fashionable knowledge stack, progressive use circumstances for metadata, and extra!

Within the first interview of this sequence, we meet Heidi Jones, knowledge evaluation and program administration extraordinaire, who explains the historical past of Docker’s knowledge crew, how they evaluated the market, and the way they’ll use Atlan to assist their colleagues drive one of many world’s finest developer experiences.

This interview has been edited for brevity and readability.

Would you thoughts describing Docker and your knowledge crew?

Docker is a platform designed to assist builders construct, share, and run fashionable purposes. We deal with the tedious setup, so builders can deal with the code.

Information professionals at Docker help quite a lot of completely different departments. So we now have a core knowledge crew with engineers and analysts, after which we even have knowledge engineers and analysts that help the most important capabilities of Docker, similar to Advertising, Gross sales, the completely different merchandise at Docker, Finance, et cetera.

A number of skilled knowledge engineers and analysts who’ve joined Docker, have solely began within the final 9 months or so. So we’ve had fairly a little bit of progress on the information crew, and are actually at that stage the place we’re making an attempt to put money into good processes. That method, our knowledge crew can be sure that everybody at Docker has the information that they should do their jobs, and may in the end assist builders do theirs.

And the way about you? Might you inform us a bit about your self, your background, and what drew you to Information & Analytics?

I believe the principle cause I’ve been drawn to knowledge and analytics is as a result of I similar to having the ability to reply individuals’s questions. 

I got here into knowledge evaluation by way of a non-traditional route. I’ve been at Docker for about six months now, however I’ve been within the knowledge house for a few decade. It began with Excel and offering insights through spreadsheets, as much as PowerBI utilizing Snowflake, that sort of factor.

So I used to be all the time an information analyst, however then additionally a undertaking supervisor. And so what I do at Docker combines each of these. The information of knowledge and the workflows required to get good knowledge and supply good insights, and likewise the undertaking administration and operations facet of it. All of it permits knowledge professionals to deal with what they do finest, which is modeling knowledge and offering insights with out being blocked by something that has to do with workflow.

What does your stack appear to be? Why did you want an energetic metadata answer?

We ingest knowledge from quite a lot of sources in a number of alternative ways, relying on the supply. After which our knowledge warehouse degree is Snowflake. Our modeling layer is dbt, that’s the place we do modeling and transformation. After which our fundamental BI device is Looker, that’s the place we do visualization and evaluation.

We have been only a one-person crew not too way back. So all of that knowledge work was on one particular person’s plate, together with documentation and understanding knowledge sources. That’s fairly a bit for one particular person.

Quite a lot of that burden has been unfold out throughout a number of individuals on the crew by now. However we’re making an attempt to maneuver away from, “Oh, let me go ask my favourite knowledge particular person,” towards, “I can go test this device and I do know there’s a licensed knowledge asset.” 

And so, due to our stack, we have been drawn to Atlan due to issues just like the Looker Chrome extension plugin, the dbt integration, that sort of factor. As a result of proper off the bat we have been in a position to say, “Okay, any descriptions we put in our dbt layer will routinely be uncovered in Atlan.” 

So non-engineering customers who need to know what the information means can go straight to Atlan and see what’s being completed within the modeling layer.

Did something stand out to you about Atlan throughout your analysis course of?

Atlan is a really cool device that has a superb suite of options that we have been on the lookout for, however the differentiator actually got here all the way down to the individuals at Atlan.

You demonstrated very competent understanding of the issues within the knowledge house and likewise very mature buyer help. We might inform that your help was not simply one thing you have been promising for us, however one thing that you just have been already actively doing with different prospects. 

We knew that it could be an actual partnership and that the shopper help org was ready to help the wants of a company like ours. And that maturity stood out to us after we made our resolution.

However then once more, additionally the options like Playbooks, the integrations that I’ve already talked about with dbt, with Looker, and simply the fixed innovation as properly that we have been in a position to observe even in the course of the analysis processes, which I consider took us about two months.

There have been a number of improvements and releases that occurred throughout that point interval and we might see the cadence the Atlan was on to repeatedly enhance. All of these have been promoting factors to us.

What do you plan on creating with Atlan? Do you’ve got an thought of what use circumstances you’ll construct, and the worth you’ll drive?

Our largest worth that we’re making an attempt to drive with Atlan is to guarantee that professionals at Docker can get the knowledge they want concerning the knowledge that they should do their jobs.

We need to transfer in direction of self-serve analytics and permit each knowledge professionals, and those that simply need to have the ability to use knowledge extra freely of their work, to have the ability to accomplish that with out having to get into all the SQL and technical particulars of the information.

They know they will belief the information set, they know they will belief the information that they’re , and so they can go forward and make their selections. In the end, it ought to assist us help our mission of delighting builders, and growing instruments that they get pleasure from utilizing.

We’ll be supporting that with Atlan, and likewise supporting our knowledge engineering and analytics groups. They should have extra supported and standardized workflows, in order that they will deal with modeling, actually digging in and doing what they do finest with knowledge.

Did we miss something?

That’s a superb query. I believe how we found Atlan was attention-grabbing. I’ve been following Prukalpa, truly, for a few years simply as an information skilled, simply type of watching Atlan.

And so after I joined Docker, they have been already knowledge catalog instruments, however hadn’t been Atlan but. And I stated, “Effectively, how about Atlan? Ought to we take a look at Atlan as properly?”

So one of many first issues I did at Docker was to start out up that dialog, and the explanation why I did that’s as a result of I had appreciated studying what she stated in these areas. Concerning the causes we’d like knowledge catalog instruments, and past only a catalog, the way it could possibly be a part of knowledge operations. And that piece of it actually had spoken to me over time. 

And we noticed some spectacular instruments. It’s a burgeoning house. There’s some nice instruments on the market. However I’m glad that we additionally checked out Atlan as a result of in the end it had a superb mixture of what we wanted at Docker.

Picture by Annie Spratt on Unsplash