Terminology

An answer contains the students files/solution for a single Task.

An assignment groups multiple Tasks. It has a title, start-date, deadline and some other meta-data. In real life it would be some form of worksheet you give to students.

Containers are isolated processes that run on Linux machines. They are always based on an Image. Containers allow secure execution of submitted code.

An Evaluation is the result of a executed Evaluation Definition on a students Answer. It consists of multiple Evaluation Steps that have been executed by Evaluation Runners.

The Evaluation Definition refers to the contents of the codefreak.yml file. It defines which Evaluation Runners should be run on an Answer and the configuration for them.

An Evaluation Runner executes the automated evaluation of source code. There are different types of runners that accept different forms of configuration. Runners are always executed in Containers.

An Evaluation Step groups all Feedback that has been generated by an Evaluation Runner by testing the submitted source code. The result of an Evaluation Step can be "successful" or "failed". The step is only successful if all Feedback is "successful".

The smallest unit in the evaluation context is Feedback. It consists at least of a single "summary" that gives some helpful feedback on an Answer. A Feedback optionally refers to a file or parts of it. Feedback is generated automatically by all evaluation runners or can be created manually by creating comments.

An Image is the boilerplate for Containers. It contains all files that form a basic filesystem and some default instructions to run in containers.

A submission groups all Answers of a single User for one Assignment.

A task contains the boilerplate code the students start working on and the Evaluation Definition.

As a teacher you might use the same Tasks over and over again in different Assignments and/or semesters. The Task Pool allows you to define Tasks once and use them in different Assignments.