Driving Self-service, Safe, Reliable Knowledge with Atlan

• Swapfiets, the world’s first bicycle subscription service, sought to launch a governance program to fulfill rising inner demand for information
• Selecting Atlan, Swapfiets launched their governance program with a self-service catalog, providing metrics and supply definitions throughout six enterprise domains
• Having efficiently launched the catalog, Swapfiets’ information workforce now routinely runs governance committees to strengthen cross-functional collaboration, and is driving a tradition of information savviness via coaching and enablement

Based within the Netherlands in 2014, Swapfiets is on a mission to make cities extra livable, introducing the world to the idea of “Bicycle as a Service” with a month-to-month subscription to a bicycle with restore companies included. Now with over 270,000 subscribers throughout eight nations, Swapfiets’ achievement of sustainability targets, and its continued progress, relies on dependable, simply accessible information.

“I’m actually keen about information. Knowledge is so numerous, and it has not solely a technical half, but additionally a governance, or extra summary half, as effectively,” shared Lisa Smits, Supervisor of Swapfiets’ Knowledge & Analytics workforce. “That’s how I got here to Swapfiets, and that’s how I got here to guide this workforce.”

Drawing upon a various skilled background spanning jobs from provide chain administration to pricing evaluation, and together with the assistance of her workforce and colleagues, Lisa has helped shepherd in a brand new period of information savviness and collaboration throughout Swapfiets, constructing and rising a Knowledge Governance apply, and making use of best-of-breed expertise like Snowflake and Atlan.

Knowledge is a crucial a part of measuring the place we are actually, and what we have to do to develop additional. We’ve got initiatives like optimizing the components we use and being extra sustainable, and that’s the place my workforce, straight and not directly, contributes to the entire firm.”

Lisa Smits, Knowledge Analytics Supervisor

As soon as structured as one giant workforce, Swapfiets has since damaged up its information operate into three specialised components. First is Knowledge Engineering, chargeable for creating and sustaining Swapfiets’ information platform and driving information high quality and availability. Subsequent, Knowledge Science creates complicated information merchandise via superior modeling, resembling demand forecasting and buyer lifetime worth. Lastly, the analytics workforce led by Lisa is chargeable for Swapfiets’ BI environments and activating information merchandise. Powering this information workforce evolution is a rapidly modernizing information stack, atop Snowflake as their information warehouse.

“We migrated to Snowflake. It’s extra versatile to scale up and down, it takes lots much less time for our engineers to arrange, and permissions administration was a giant factor that was simpler to do. This transitioned us into being a extra mature information workforce,” Lisa shared.

Swapfiets additionally adopted Fivetran, automating the pipelines as soon as run on guide scripts, and dbt, using their semantic layer as a decentralized metrics retailer. Rounding out this modernization was the adoption of Airflow and Tableau.

“We matured in a single huge step and actually specialised,” Lisa defined. “We targeted on having instruments to purchase, versus construct, making a acutely aware resolution on paying a bit extra, however saving a lot time in the long run, leading to decrease prices.”

Whereas Swapfiets was maturing considerably throughout technical and organizational domains, tribal information about their information belongings endured. Initially, the workforce created a Google Sheet with an inventory of 25 metrics and definitions. However with no easy interface to entry these metrics, or perceive the context round them, the amount of questions posed to the Knowledge & Analytics workforce elevated.

“We bought numerous questions from individuals, like ‘How is that calculated?’ or ‘We see this quantity on this dashboard, however on this dashboard, it’s fully totally different. How does that work?’,” Lisa defined. “There was no possession, no transparency, and no central place for us to write down down phrases, or clarify what metrics meant and the way they have been calculated. It took numerous time to reply these ad-hoc issues.”

Most often, there was a superbly legitimate motive for variations between dashboards and belongings, however absent documentation, belief was eroding, main Lisa and her workforce towards a brand new deal with information governance.

To lose belief is simple, however to realize that belief again is tough. That’s once we began taking this severely. We wished to have a high-quality, reliable BI surroundings, and that’s why we began trying into our choices for governance, and into Atlan.”

Lisa Smits, Knowledge Analytics Supervisor

Lisa and her workforce started by looking for glossary options with a view to enhance the transparency of their information belongings and cut back the amount of questions from Swapfiets’ information customers. However as she and her workforce realized extra in regards to the Lively Metadata Administration market, their expectations for the worth that such a platform may drive grew.

With a versatile person interface, information customers may uncover, perceive, and apply information in a self-service method, and with automated lineage, Swapfiets’ information practitioners may higher perceive how information flowed via their modernizing stack and the way it was being utilized. And with Atlan Insights, a metadata-based question builder, much more of Lisa’s colleagues could be empowered to make the most of information.

Quite than only a glossary of phrases, Lisa and her workforce envisioned a one-stop store for his or her colleagues who used information day-after-day, and for many who ought to, however hadn’t but.

You could have one place the place you possibly can retailer the phrases, the place you possibly can take a look at your entire information belongings, the place you’ve got the descriptions, and the place you possibly can see the house owners. When you have a query, you possibly can instantly ship it through Slack. It’s all related in a single place, and I feel having that full image, not solely governance but additionally self-service, was what actually stood out to us about Atlan.”

Lisa Smits, Knowledge Analytics Supervisor

After selecting Atlan, Lisa and her workforce started by integrating Redshift (since migrating to Snowflake), dbt, and Tableau, rapidly enabling lineage throughout their most crucial applied sciences. With their information property successfully crawled, and information belongings seen, they started the method of assigning house owners and making a enterprise glossary.

Utilizing the definitions they’d already agreed upon with their enterprise colleagues and had saved in Google Sheets, Lisa’s workforce enriched a baseline of a brand new enterprise glossary, then labored with their colleagues in Knowledge Engineering to doc the sources of every asset. Whereas utilizing pre-existing definitions saved useful time, the method of enriching information belongings and documenting their supply was nonetheless a cumbersome one. In Atlan Playbooks, rule-based bulk automations for duties like documentation, Swapfiets discovered a solution to considerably speed up enrichment.

Using Playbooks, Swapfiets constructed an automation making certain that every time a brand new asset is launched into their metrics retailer, its pre-existing descriptions from upstream techniques are robotically handed via to Atlan. These descriptions are then verified, slightly than written from scratch, considerably accelerating their documentation rollout.

“Content material creation was numerous work, however whenever you launched Playbooks, it was superb for us,” Lisa shared. “We now have 125 metrics, and it improved our time spent on documentation.”

Furthering their use of Playbooks, Lisa’s workforce rapidly moved to safe these newly obtainable information belongings, robotically tagging delicate or personally identifiable data, and utilizing personas inside Atlan to make sure delicate information was obtainable solely to these licensed to view it, or contribute to documenting it.

Searching for to deepen collaboration with material specialists, and to make information governance a routine a part of the way in which Swapfiets operates, Lisa attended an Atlan Masterclass with WeWork to find out how greatest to construct and preserve a enterprise glossary.

Lisa’s workforce maintains a single enterprise glossary, divided into enterprise domains that align material specialists with companions on the information workforce. Six such enterprise domains exist inside the glossary, throughout groups like Operations, Human Sources, Product, and Client Direct. And for every of those six enterprise domains, Lisa’s workforce organized cross-functional governance committees composed of information analysts, information house owners, and information stewards that meet on a month-to-month foundation to debate their progress.

“In that month-to-month assembly, we deal with three governance subjects, that are possession, documentation, and high quality. For documentation and possession, Atlan is actually helpful. It’s additionally helpful for high quality to search out out the place within the information lifecycle high quality points come up,” Lisa shared. “Throughout the assembly, we write out motion factors and decide to duties like verifying extra phrases.”

To trace progress in opposition to these commitments, Lisa’s workforce makes use of Atlan’s reporting performance as a heads-up show of what number of phrases and information sources have been verified or enriched, and at what tempo. Additional simplifying the method, material specialists use filters in Atlan to show the phrases they’re chargeable for, giving them an inventory of related phrases and information sources they’re tasked with enriching, or to mark unverified however necessary phrases and information sources as draft for future enrichment.

Whereas Swapfiets continues to search out new methods to use Atlan and develop their governance apply, the worth yielded from self-service and their metrics catalog, alone, has been vital.

“Colleagues steadily ask ‘What does this imply?’ or ‘How do you outline this metric?’ With an built-in workflow in Slack, we ship them to the Atlan glossary to self-serve their query. It’s been a good time saving for us,” Lisa defined.

For questions that may’t be answered by self-service, the Knowledge & Analytics workforce advantages from clear, assigned possession for every information asset, making it easy to direct inquiries to material specialists, or to tag them straight right into a Slack thread.

By standardizing the method for outlining metrics, and automating their preliminary enrichment utilizing Atlan Playbooks, Lisa’s workforce has saved many hours of effort, to this point. However extra noteworthy to their workforce is the truth that they’re now assured their course of will probably be adopted every time a brand new metric is launched.

The entire standardization of all of the metrics saves between 10 and half-hour for each metric. So should you multiply that by 125, it’s a pleasant quantity, nevertheless it’s not like, ‘Oh, we saved 5 weeks this 12 months.’ It’s simply good that we don’t have to consider this anymore. It’s not in our course of anymore whereas we create a metric. It’s already taken care of.”

Lisa Smits, Knowledge Analytics Supervisor

With a catalog of metrics in place, and a sustainable course of to additional enrich it, Lisa and her workforce started a sequence of packages designed to extend information savviness throughout the Swapfiets group and additional cut back the burden of service on their group.

Knowledge Literacy Program

Foundational to how Swapfiets permits self-service is their Knowledge Literacy Program. A part of this program is a compulsory coaching for Atlan & Tableau. This teaches new Swapfiets workers about Atlan and its prospects, and the right way to make the most of Swapfiets’ present Tableau dashboards. When full, customers are then given entry to Atlan and Tableau. On this program, the Analytics workforce additionally affords a SQL-Atlan coaching for his or her Knowledge Champions.

For self-service questions, we like educating individuals the fundamentals of querying. And with Atlan, you’ve got the visible question builder. Even when individuals are not tremendous information literate, they’ll mess around with the question builder and it’s a lot simpler to grasp than one thing like ‘SELECT * FROM’. If they should filter, it’s really known as filter as a substitute of the technical ‘WHERE’ assertion. It simply speaks to them much more.”

Lisa Smits, Knowledge Analytics Supervisor

With the mixture of the Knowledge Literacy Program and Atlan Insights, the usage of Tableau is now reserved for long-term information merchandise and visualization, with the majority of customized querying now completed utilizing Atlan, probably reducing licensing prices over Tableau and reducing the technical limitations to information evaluation with a visible question instrument.

Knowledge Champions

For Swapfiets workers with a better technical aptitude or ambition, or for information stewards chargeable for updating their glossary, a extra superior degree of coaching dives deeper into visualization in Tableau, and the right way to discover belongings and translate them into queries in Atlan.

Utilizing fine-grained entry controls in Atlan, Lisa’s workforce ensures these Knowledge Champions are capable of conduct self-service evaluation at a deeper degree, with out the chance of accessing and making use of improper or restricted information.

“We don’t have any PII in Tableau anymore, as a result of entry administration is simpler in Atlan. Now, it’s clear what number of belongings have PII information, who can entry these, when they’re used, and who queries which tables,” Lisa shared.

Extract Champions

The latest addition to Swapfiets’ democratization efforts are Extract Champions, data-savvy colleagues from every enterprise area that are actually empowered to assist Lisa’s workforce when extract requests for information happen.

In Swapfiets’ information assist channel on Slack, customers provoke a workflow to request information that’s unavailable in Tableau, or too superior to self-service. With one click on of an emoticon, Extract Champions from the related information area are tagged, settle for the duty, and start querying the information utilizing Atlan Insights for supply to the requester.

“After they’re completed, they simply click on the ‘End’ button on the ticket, and we don’t even have to take a look at the ticket. So it’s a win-win. I’m so obsessed with this, it’s been nice,” Lisa shared.

Furthering the worth Extract Champions yield from Atlan, Lisa and her workforce realized the repetitive nature of some requests, and labored to automate them utilizing Atlan Scheduled Queries.

“Atlan helped us schedule extracts to individuals outdoors of Atlan, which helped an Extract Champion who’s answerable for communication with B2B clients,” Lisa defined. “And now he’s scheduled his guide work, and Atlan automated what he would normally share through electronic mail. He doesn’t have to succeed in out to us anymore.”

Sustaining the unimaginable momentum of Swapfiets’ Knowledge Governance Program is a deal with change administration, fixed communication, and driving cultural change.

Governance Committees proceed to fulfill on a month-to-month foundation, making certain every enterprise area retains their enrichment and possession obligations top-of-mind. Updates and newsletters are routinely despatched to stakeholders, educating them on subjects like high quality, possession, safety, or accessibility, and updating them on essential subjects like how documentation is progressing in opposition to their targets.

However probably the most noteworthy change, past how rapidly Swapfiets’ expertise has modernized and the way effectively their Knowledge Governance Program has been stewarded, is a cultural shift. The place Lisa’s workforce was as soon as chargeable for answering questions on information and connecting the dots for information customers, that possession is shifting. Stewards and Knowledge House owners from throughout their group now really feel empowered to discipline questions on their information, and chargeable for the standard and availability of those belongings.

We’ve got the information assist channel, and we may reply most questions, however we gained’t anymore. We tag the information proprietor or steward and they’ll look into it. This ends in extra motivation to search for sustainable options and take into consideration Knowledge Governance once they’re taking part in their half contributing to Swapfiets’ mission. I feel it’s created a tradition the place information is one thing that impacts all of us, and never simply the information workforce. That’s been the most important change we’ve completed, to this point.”

Lisa Smits, Knowledge Analytics Supervisor

Picture by Ernest Ojeh on Unsplash