Terminology

An answer contains the students files/solution for a single Task.

An assignment groups multiple Tasks. It has a title, start-date, deadline and some other meta-data. In real life it would be some form of worksheet you give to students.

Containers are isolated processes that run on Linux machines. They are always based on an Image. Containers allow secure execution of submitted code.

An Evaluation is the result of a executed Evaluation Definition on a students Answer. It consists of one or multiple Evaluation Steps.

The Evaluation Definition refers to the contents of the codefreak.yml file. It defines which Evaluation Steps should be run on an Answer and the configuration for them.

The definition of an evaluation step is the blueprint how the Answer of a student will be tested. Mainly, it consists of a bash script that will be executed, and a report parser which will interpret the script’s output.

An Evaluation Step groups all Feedback that has been generated by an execution of the evaluation script. The result of an Evaluation Step can be "successful" or "failed". The step is only successful if all Feedback is "successful".

The smallest unit in the evaluation context is Feedback. It consists at least of a single "summary" that gives some helpful feedback on an Answer. A Feedback optionally refers to a file or parts of it. Feedback is generated automatically by an Evaluation Step.

An Image is the boilerplate for Containers. It contains all files that form a basic filesystem and some default instructions to run in containers.

A submission groups all Answers of a single User for one Assignment.

A task contains the boilerplate code the students start working on and the Evaluation Definition.

As a teacher you might use the same Tasks over and over again in different Assignments and/or semesters. The Task Pool allows you to define Tasks once and use them in different Assignments.