A Framework for Scheduled Tasks

Nov 30, 2023·5 min read

Tasks are typically pieces of work (or functions) that happen on a predetermined schedule, to support higher order behaviours. There are a myriad of ways to set this up, and it is a common feature of many frameworks on the web or otherwise. In the rest of this write up I’ll explore a proof of concept I worked, called Cloc and some of the guiding principles.

Managing recurring tasks in both Golang and NodeJS, certain patterns have emerged over the years, even while operating on a small scale. Considering this, it seems opportune to consolidate these patterns into a framework which would guide users towards best practices while providing convenience.

Let’s start with a Task

Tasks are repeatable work units that occur at predefined intervals. They can vary significantly in complexity, from simple actions like checking a site's status every few minutes, to more complex tasks such as synchronizing systems or preparing data for future use. From a business perspective, tasks can differ greatly in their importance and may depend on the success of others. The key takeaway here is that tasks can be leveraged in many different ways.

From a developer's perspective, the best experiences occur when the definition of the task is independent of the surrounding system. In some instances, a major overhaul is necessary. However, with Cloc, I estimate that most of the time, a write-and-forget approach suffices. With all this in mind, what’s the bare minimum you need for a task?

export default {
  interval: 1000,
  fn: async ({ logger }) => {
    logger.info('hello from cloc!');
  },
} as Cloc.Task;

There's a significant amount of functionality built in, which allows you to focus more on your objectives rather than the process of achieving them. However, striving for a minimalist interface can lead to substantial trade-offs in the future. Cloc aims to be flexible enough to support some of the most common requirements I've encountered and has a strong foundation for future extension. Tasks can be scheduled using a numerical interval in seconds, or with an interval function that lets you dynamically determine the interval.

export default {
  interval: () => {
		if (isDaylightSavingsTime()) {
			return 4 * 60 * 1000;
		} else {
			return 5 * 60 * 1000;
		}
	},
  fn: async ({ logger }) => {
    logger.info('hello from cloc!');
  },
} as Cloc.Task;

Even with both configuration options, calculating time intervals can still be challenging. Cron expressions, a popular standard for time intervals, provide a wide range of options for scheduling tasks.

export default {
	// at 01:05am, 01:20am, 01:05pm and 01:20pm
  cron: '5,20 1,13 * * *',
  fn: async ({ logger }) => {
    logger.info('hello from cloc!');
  },
} as Cloc.Task;

To simplify the process of writing Tasks, Cloc introduces some global types into the environment. The Cloc.Task ensures type safety when writing Tasks, including the dependencies provided by your application. The logger used in our previous examples is not provided by Cloc, but is an application-specific dependency injected by the framework. In theory, you could pass any third-party library, utility, or application service as dependencies to be used across all your Tasks.

Setting Up Dependencies

For the best experience in writing Tasks, you can add your custom dependencies to the global Cloc namespace. After adding them, all cloc-provided types will be updated to ensure appropriate completions and type guards.

import { PrismaClient } from '@prisma/client';
import { Logger } from './logging';

declare global {
  namespace Cloc {
    interface Dependencies {
      logger: Logger;
      prisma: PrismaClient;
    }
  }
}

Now that everything is typed correctly, it's necessary to set up dependencies for Cloc to recognize. These can be established in any preferred file, just ensure to reference it relative to where the framework is initialized.

import { PrismaClient } from '@prisma/client';
import { Logger } from './logging';

export default {
  logger: new Logger();
  prisma: new PrismaClient();
} as Cloc.Dependencies;

Providing global dependencies to every single task has some drawbacks. Resource management, observability and debugging can become challenging if all tasks share the same resources. To address this issue, the framework offers a transformDependencies method. This method can replace global dependencies with scoped counterparts for each task. For instance, a 'hello' task receives a 'hello Logger', while a 'goodbye' task gets a 'goodbye Logger'.

function transformDependencies(
	task: Cloc.NamedTask, 
	dependencies: Cloc.Dependencies
) {
	return {
		...dependencies,
		logger: dependencies.logger.createChild({
			namespace: task.name
		})
	}
}

Putting together an Example

Once all the components are ready, our final file structure should resemble the following.

.
├── app.ts
├── dependencies.ts
├── logging.ts
└── tasks
    └── hello.ts

Most of these files have already been discussed in detail, except for app.ts. In this entry file, everything is integrated using the method provided by Cloc.

import { cloc } from '@ededejr/cloc';

 async function main() {
  cloc({
    tasksPath: './tasks',
    dependenciesPath: './dependencies',
  });
}

main();

This marks the completion of the setup for a robust and adaptable task management system. The framework is engineered to accommodate various needs, prioritizing simplicity and convenience. Wishing you a productive coding experience.

Resources