|Gator is a framework for running a hierarchy of jobs and aggregating logs, metrics, resource utilisation, and artefacts.|
Gator is compatible with Python version 3.8 through 3.11 and is developed using the Poetry packaging tool. To install Gator, check out the repository from GitHub and install it with Poetry:
$> git clone [email protected]:/Intuity/Gator.git $> cd Gator $> poetry install
Before running Gator, you will need to create a job specification as described below. Then invoke the tool as follows:
--progress switch enables a progress bar which tracks jobs as they
running and the number of passes and failures.
The complete set of tasks to run are specified with three objects:
- !Job - specifies a single task to run along with the working directory, environment variables, and required resources;
- !JobGroup - groups a collection of different jobs together into a named tier;
- !JobArray - similar to a !JobGroup but it repeats the group of jobs multiple times.
Jobs, groups, and arrays can form dependencies on one another, allowing jobs to be sequenced and only start when a previous job completes (either with success or failure).
A simple example of the syntax can be seen below:
Project mascot: Alligator by TRAVIS BIRD from the Noun Project
A lot of the ideas in this project have been taken from the experiments detailed on Rich Porter's blog from 2013 and 2014.