A general purpose architecture for background processes

Introduction

The purpose of this article is to present a general purpose architecture for background processes. In this page, background process is used interchangeable with job.

Like any general purpose architecture, it cannot fullfil every single case since the requirements can be different across the different applications.

This architecture tries to address the requirements listed below and it should be adjusted accordingly whereas the specific case has a different set of constraint, requirements and wishes.

Requirements

In order to have a general purpose architecture that can be applied in a high number of scenarios, the following requirements should be satisfied.

Separation of concern

The business logic performed by the job must be loose coupled from the launcher logic.

The laucher component is responsible just to invoke a specific job.

The launcher can be either a scheduler that automatically triggers the job according to its configuration or manually through an explicit trigger action

Multiple Execution Environment

A general purpose architecture should allow to run the jobs either in a standalone environment or in a container.

Cluster awareness

In case jobs are deployed and launched within a container (e.g. within a web application), they must support those scenario where the deployment is performed in a cluster environment.

If a scheduler is used as a launcher component to trigger a job, it must keep in account that the job, according to its scheduling, must be triggered only once and not multiple times because of the multiple nodes.

 Description

The architecture is based on a launcher (trigger) component that is responsible to run the job.

Since there could be different way how a job can be launched, the launcher component should contains multiple implemetations such as

  1. REST API: If the launcher is running in a container a REST endpoind or any other protocol could be used to trigger the job remotely
  2. Command Line Interface (CLI): A command line interface could be useful to trigger a job (either on a local machine or remotely) using shell commands
  3. Internal Java API: Once the launcher component is deployed along with an enterprise application, it could be useful to expose the launcher API so that other Java components will have the capability to trigger jobs

All different launcher implementations finally could rely on the popular Quartz Scheduler to actually trigger a job.

Among many other features, it supports deployments in cluster environments where the scheduler runs on multiple nodes but still the jobs are triggered only once.

The job itself can be either a typical Spring Batch job or any generic background process.

GeneralPurposeArchitecture

 

 

 

 

 

 

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.