December 19, 2023 | 6 min

Modern Data engineering with software best practices

Jelle Vegter
Head of Data Management

With more transactions than the combined card transactions of all the Dutch banks, Data Engineers at Nederlandse Loterij are faced with rare challenges in the Dutch data landscape. This, of course, requires a strong vision. In this blog, Head of Data Management Jelle Vegter talks about Nederlandse Loterij’s data strategy, its data foundation, and the potential of real-time data applications within the company.

 

From Marketing to MarTech

It’s not every day that a new company is created within an existing one. This is precisely what happened when the online gambling market opened up in October of 2021. To support the far-reaching changes that come along with offering iGaming products, a cultural shift was initiated on an organisational level; from being a Marketing to a MarTech company. This transition is essential to meet the new, technologically complex challenges of offering games 24/7. “These technical and organisational challenges were the main reason for me to join Nederlandse Loterij.”

 

Data strategy

All the way up to the executive board level, Nederlandse Loterij sees data as essential for its success. “Nearly everyone is recognising the potential of the data. The intelligent use of data is reflected in almost every departmental plan. Our data strategy therefore primarily revolves around creating a data-driven ecosystem, in which analysts from every department can quickly start working with data to generate value. Nederlandse Loterij’s data strategy focuses on democratising data and making data accessible.”

“Per dataset, we determine its real-time requirements. For example, we are currently monitoring triggers in real time for the Responsible Gaming department. This enables us to quickly send notifications to players if we see unadvised gaming behaviour.

We have countless ideas of applications of (real time) data. However, it’s crucial to ensure the data products we build add more value than they cost. A constant focus on business value ensures we build sustainable products.”

 

A robust data foundation

The importance of a strong data foundation cannot be underestimated, especially in the current technological landscape. With the increasing focus on artificial intelligence from mainstream media (e.g., ChatGPT), it’s clear that many companies are starting to realise that they don’t have the capabilities to make the most of it, simply because they haven’t got their basics in order. “All the cool, shimmering gems – like fancy dashboards, impressive algorithms, and bots that watch everything – only provide real value if the hygiene factors are sorted out. One way to achieve this is to treat Data Engineering with the same rigorous principles as software engineering. This means applying software best practices to the data landscape. As far as Jelle is concerned, key methodologies are:

  1. Version control
  2. CI/CD
  3. Automated testing

 

“In addition to adopting these best practices throughout our data ecosystem, we’ve consolidated the architecture of our environment by assigning specific responsibilities to the various data products we’ve built. The systems are then integrated to provide synergy and reduce duplicate efforts. We have systems specialized in (aggregating) real-time data, big data & AI, and analytics.” From the Data Management department, Jelle and his team ensure each application serves its intended purpose while abstracting the complexities behind the scenes. “Our users will experience one unified platform.”

“The tremendous amount of data, combined with multiple real-time applications, and the need for near-perfect data quality for compliance, makes the work very challenging.”

Jelle Vegter
Head of Data Management

 

Engineering culture

Working at Nederlandse Loterij means working in an agile work environment. Jelle’s department consists of four specialised data product teams. “These teams consist of 3 to 4 Engineers, 1 Tester, 1 Analyst, and 1 Product Owner. The balanced composition of these teams allows them a high degree of autonomy, as they can be largely self-steering. My main role is to align our data activities with overarching business objectives and set the long-term strategy. Within this setup, the teams have a lot of freedom to fill in the ‘how’ and build innovative, cool things that provide real business value. At Nederlandse Loterij, people are given both trust and responsibility.

 

Technical challenges

The technical challenges within Nederlandse Loterij are complex. Since the iGaming market went live at the end of 2021, the volume of the data has surged dramatically, with more than a billion events are processed monthly. This tremendous amount of data, combined with multiple real-time applications, and the need for near-perfect data quality for compliance, makes the work very challenging. And interesting!

“To meet these challenges, we use cutting-edge technology and are constantly looking for ways to make the environment more efficient, faster, more accurate and cost-effective. Fortunately, we are not on our own. We work closely with partners, such Microsoft and Databricks. We regularly hold joint design and brainstorming sessions to guide and adjust ongoing architectural changes.”

 

Data & AI

Some of the topics you’ll encounter at Nederlandse Loterij when working in data are compliance, anti-fraud, responsible gaming, and personalisation. These themes offer excellent opportunities to leverage streaming data with AI to make a significant impact. For instance, fraud can be detected in real time based on placed bet combinations and non-responsible gaming behaviour can be alerted on based on gameplay tendencies. “At Nederlandse Loterij, we are committed to providing games of chance responsibly. We believe we can make a significant contribution to this mission with the intelligent use of data.”

To realise all these ideas, we’ve built a future-proof platform that’s up to the task. “Our data ecosystem allows us to train AI models with rich data sets. We have an application landscape with real players. For our models to truly have an impact, we aim to integrate the data environment more closely with the application landscape to facilitate interaction. This allows us to train models and then, from within the application landscape, make inferences to the models. Real-time data interaction empowers us to act proactively rather than reactively.”

Think it’s cool to work for an organisation with a unique data landscape? Check out the Data Engineer vacancy here.

Stories

Team players tell
their stories

All the unique talents of our colleagues come together in various teams. In this way, we work together on a happy, healthy and sporty Netherlands. Want to know who’s on your team?

Lees onze verhalen
Using data to make an impact at all levels of the organisation

Using data to make an impact at all levels of the organisation

Get to know us even better