Energinet; From humble beginnings to an enterprise Grafana setup!
Energinet is a Danish state-owned energy company that works to ensure a secure and reliable energy supply for Denmark. It operates and administers the electricity and natural gas transmission networks in Denmark. It also works to develop a competitive energy market in Denmark and to promote renewable energy sources. Additionally, the company is involved in several projects in the energy sector, both in Denmark and abroad, to develop new solutions for the future energy system. Energinet is at the frontiers of monitoring their assets.
Energinet, in collaboration with HCS Company, have built a test and production server with Grafana. What were the technical issues and what are the advantages? Lena Kamstrup Kjær, the project manager and business developer at Energinet, and Benoit Schipper, the SRE at HCS Company, will tell you everything about it.
Opdrachtgever: Energinet
Sector: Climate, Energy and Utilities
Diensten: Consulting
Single point of failure
We’re zooming in at the Asset Management department. This department is responsible for collecting data to maintain and predict asset maintenance. It all started when one high-voltage engineer with plenty of expertise installed Grafana on a self-managed server. Grafana allowed him to easily transform the data into graphs that could be understood by everyone. While a singular Opensource setup can get you far, there was a problem: what if the system failed or the engineer running it drops out? They would be left with nothing. In other words, there was a single point of failure.
What was needed?
Energinet needed to move away from the initial Grafana setup and professionalize even though it has served them well. When we asked Lena about their needs, she mentioned the following requirements:
- They needed the IT department for Operations, Support & Maintenance.
- They needed to solve the issue surrounding the single point of failure.
- They needed a professional and supported solution.
- They needed tangible results before the end of the year 2022.
These requirements are needed to receive widespread support to further invest time in Grafana, allowing other teams and departments to also jump on the Grafana train.
Energinet and HCS Company
It all started when Benoit and Henrik (the Engineer who introduced Grafana) met at an Opensource Grafana event in Amsterdam. Due to Energinet being a state-owned company, Lena set up a European Tender requesting Grafana consultants. After a selection process, HCS Company emerged victorious and was chosen to help Energinet succeed.
“HCS was chosen due to their knowledge, flexibility and the immediate “click” we established with one another.”
Lena Kamstrup Kjær
The approach
Together we naturally agreed upon an approach we both believed in. The IT department of Energinet set up two virtual servers that would be managed by them. They would be responsible for operations, support and maintenance including the installation of Grafana. HCS Company and Lena’s team would be responsible for migrating away from the self-managed server that was managed by Henrik. The IT Department provided a Test and Production server and handed over the keys.
HCS Company decided to use what was already available within Energinet and decided upon creating a tool in Python and use Grafana’s existing API endpoints. The command-line tool would allow the user to point at a Grafana Server, extract all the Dashboards, Datasources and many other configuration files in JSON-format. We could then either save these configuration files in GIT or push them directly towards any other Grafana instance we had access to. This made it possible to move any Grafana configuration around in bulk and allowed us to speed up this otherwise manual process. This gave us more time to focus on perfecting the Grafana setup on the test server to prepare for the migration towards the production server.
The result
By flying over two HCS Engineers from the Netherlands (Stein & Benoit) and focus on the task we were able to achieve this massive feat within 4 working days in mid-December 2022. This could only be achieved by combining our forces. HCS Company worked closely with Henrik and Lena's team. Energinet supplied the domain knowledge and HCS Company the IT knowledge. Benoit: “You could call it something like co-building. Together we acted like brick and mortar.”
Soon we will be moving towards production together. From this point on enabling and supporting the business to use this solution becomes the most important thing. HCS will continue to assist Energinet in exploiting this success surrounding Grafana.
HCS and Grafana now allow us to take full advantage of all the data that is available to us. It has become a platform that fosters curiosity and agility that allows for better data insights.”
Lena Kamstrup Kjær
