Why AI Architecture Differs from Traditional Applications
AI systems have specific requirements that distinguish them from typical web applications. Language models require significant computational resources, but not always and not everywhere. Different system components have diametrically different load profiles: the GPU inference module is the resource bottleneck, the API layer must handle thousands of concurrent requests, and training processes run in batches and can take hours. The architecture must account for this.
Arguments for Starting with a Monolith
Most AI projects should start as a monolith. The reasons are pragmatic: monolithic architecture is simpler to debug, easier for new team members to understand, and faster to iterate on. When the system is in an experimental phase and component boundaries are still forming, premature decomposition into microservices causes more harm than good.
A classic mistake is designing dozens of microservices before the system serves its first real user. Unknown usage patterns translate into incorrect service boundaries, which then generate enormous refactoring costs.
When Microservices Are the Right Choice
Migration to microservices makes sense when specific, measurable problems with the monolith emerge:
- Different scaling requirements — the OCR module needs 10x more resources during campaigns, while the rest of the system remains stable.
- Independent deployment cycles — the ML model team deploys new versions several times a day, while changes to the reporting module happen once a month.
- Fault isolation — a bug in one component should not interrupt the work of others.
- Different technology stacks — the image processing module requires Python libraries unavailable in the main stack.
Migration Strategies from Monolith
Safe migration follows the Strangler Fig pattern: gradually extracting functionality from the monolith and replacing it with independent services, while keeping the system running at all times. The key is identifying natural domain boundaries — not artificial cuts along technical lines, but divisions aligned with business logic.
The first candidate for extraction should be a component that: is well-defined functionally, has a clear and stable API, and generates the most problems due to differences in scaling requirements. Typically, this is the AI model inference module.
Multi-Agent Orchestration and Service Architecture
ESKOM.AI multi-agent systems operate on a layer above traditional microservices — individual AI agents can leverage both a monolithic business application and a microservices ecosystem. The key is designing interfaces so that the target system's internal architecture does not limit integration capabilities with agent logic.