Architecture Performance Overhaul

Note: due to NDA I cannot show the real UI or reveal the client name

• Employer: NTT Data
• Client: Energy Company
• Role:: Lead Architect
• Duration: 3 months
• Technologies: Angular 16, Java, REST, SQL

Project Overview

A web app showing real-time data about energy distribution, consumption and monitoring of the energy network of Spain managed by this specific company. This app allowed users to visualize data in collapsible tables, charts and maps, it was also handling a continuous flow of information and massive amounts of data.

The Problem

The huge amount of data ingested by the app were significantly affecting the system’s performance on all levels, the UI was extremely laggy and unresponsive, a single server request would often take more than 30 seconds to complete and huge payloads of hundreds of megabytes would be downloaded by the application, some pages would put a very heavy load on the system CPU, often consuming more than 2-3 GB of RAM per browser tab.

The Solutions

This project was extremely interesting as it required a multidisciplinary approach and an extensive analysis of the entire system architecture, both on a high level and down to the code implementation of single UI components.

After the initial analysis phase, I designed the technical solution, prepared a plan of execution and helped the engineers writing the code and implementing the architectural changes. Some of the things I did were:

• Upgrading the server protocol to HTTP/2 to increase performance
• Implement virtual scrolling for very large tables.
• Leverage REST APIs content negotiation to use a more efficient format for APIs
• Enable server side compression of the data
• Use Server-Sent events to stream real time changes to the UI
• Optimize the database driver’s configuration to speed up queries
• Use more efficient algorithms to manipulate large data sets
• Create materialized views on the database to improve query efficiency

Results & Conclusion

The impact of my solution was astounding as we immediately noticed significant improvements on all sides of the application:

• The network payloads of large requests dropped by 90% (from 120MB down to ~10MB)
• The initial load speed was reduced by around 70%
• the users were able to smoothly navigate tables with hundreds of thousands rows.
• The slowest query speed dropped from 30 seconds to less than 2 seconds
• the app responsiveness drastically improved and UI hiccups disappeared

A very interesting project that taught me about the importance of a multi-disciplinary approach at problem solving. This project challenged me in all technical areas of web development (frontend, backend, database and infrastructure), and the significant performance improvements speak by themselves.

---