Definition Files
In Code FREAK, assignments and tasks can be defined YAML files. This includes the configuration of automated evaluation.
1. Assignment
An assignment is defined by a directory (or packaged archive) containing a codefreak.yml
with the following schema.
Field Name | Type | Description |
---|---|---|
title |
|
REQUIRED. The name of the assignment. |
tasks |
|
List of paths to task definitions (relative to the assignment). |
title: My Assignment
tasks:
- my-task-1
- my-task-2
├─ codefreak.yml ├─ my-task-1 │ ├─ codefreak.yml │ ├─ main.c ├─ my-task-2 │ ├─ codefreak.yml │ ├─ ... ├─ ...
2. Task
A task is defined by a directory (or packaged archive) containing source code files and a codefreak.yml
with the following schema. The codefreak.yml
will be excluded from the source code files.
Field Name | Type | Description |
---|---|---|
title |
|
REQUIRED. The name of the task. |
description |
|
Detailed instructions for students. Basic markdown is supported. |
hidden |
[ |
Patterns of files that should be hidden from students. Matching files are only included for evaluation. If matching files are created by students, they are ignored for evaluation. |
protected |
[ |
Patterns of files that should be read-only. Students will be able to see matching files but modifications are ignored for evaluation. Non-existent files can be protected to prevent their creation. |
evaluation |
Map[string, Evaluation Step Object] |
Map of key to an evaluation step object. They key acts as unique identifier for the evaluation step within this task. |
File patterns use the Ant pattern syntax. For more information refer to the official documentation. |
title: My Task
description: Write a function `add(int a, int b)` that returns the sum of `a` and `b`
hidden:
- src/test/**
- .codeclimate.yml
protected:
- build.gradle
- settings.gradle
evaluation:
my-step:
title: My first check
script: |
echo "Hello World"
report:
format: default
3. Evaluation
Automatic evaluation consists one or more steps Each step consists of a script that evaluates the code and a report parser, that will parse the result into a list of feedback. An evaluation step is defined as follows.
Field Name | Type | Description |
---|---|---|
script |
|
REQUIRED. A (bash) script that will be executed for evaluation. |
title |
|
An optional human readable name for the evaluation step. Summarize briefly for your students what this evaluation step checks (e.g. "Test functions.py", "Class XYZ Tests", "Code Style", …). |
timeout |
|
An optional custom timeout in seconds after which the evaluation step will be canceled and marked as failed. There is a system-wide default timeout of 5min. |
report.format |
|
REQUIRED. The id/key of one evaluation report parser that will try to parse the result into a list of feedback. See Report Parsers for a list of currently supported parsers. |
report.path |
|
A path or matching-pattern for a file that will be read by the report parser. If a blank path is specified the console output will be passed to the report parser. |