In this chapter you will learn how to create an assignment. We will create a simple Python application with a single function
add(a: int, b: int).
This chapter covers the following:
How to create an assignment and tasks
How to provide a basic file boilerplate to your students
How to define evaluations for your tasks
Running your tests locally
Preparation is key! Don’t start working with Code FREAK if you haven’t thought about what to test and how to test. Before you start adding Code FREAK configuration to your tasks, make sure they are working/compiling locally on your machine! You can also test execution like Code FREAK does locally by installing Docker. More on this in section Testing locally with Docker.
For this tutorial we use the following
main.py Python file with a single function
add(a: int, b:int) → int. You can either let your student create these file or at least provide some initial template (boilerplate) to give them a fair starting point.
def add(a: int, b: int) -> int: # TODO: write the algorithm to add a and b return 0 (1) if __name__ == "__main__": print("2 + 3 = %s" % add(2, 3))
|1||This is the line the student has to correct to finish this task.|
Which form of testing you choose for your tasks heavily depends on the tasks requirements. If students have to work on algorithms you should use some form of unit or functional testing. If they create a simple I/O application you can use a library that tests input and output.
In this example we will use pytest, a testing framework for Python. It is important that your testing framework supports jUnit XML output so Code FREAK is able to parse the results.
main.py file we will create a
main_test.py file that will contain some assertions to verify the
from main import add def test_function(): assert add(2, 3) == 5 assert add(-1, 2) == 1
You should verify that this works as expected on you local computer before continuing. We run the tests by executing the following command in the terminal.
This should give some output about a failing test. It is important that your tests are failing. Otherwise you could not expect your students to fix the faulty source code.
==================== test session starts ===================== platform linux -- Python 3.8.2, pytest-5.4.1, py-1.8.1, pluggy-0.13.1 rootdir: /home/user/python-example collected 1 item main_test.py F [100%] ========================== FAILURES ========================== _______________________ test_function ________________________ def test_function(): > assert add(2, 3) == 5 E assert 0 == 5 E + where 0 = add(2, 3) main_test.py:4: AssertionError ================== short test summary info =================== FAILED main_test.py::test_function - assert 0 == 5 ===================== 1 failed in 0.04s ======================
You should already have a
main.py file and a
main_test.py file. Transform your directory layout to match the following.
/ (1) ├─ codefreak.yml (2) ├─ task-1/ (3) │ ├─ codefreak.yml (4) │ ├─ main.py (5) │ ├─ main_test.py (6)
We now have to add two configuration files to the directory-layout, so Code FREAK knows how to evaluate the task. As
explained in the previous section there are two different
codefreak.yml files: One for your assignment and one for
Please see the Definition Files Chapter for a full reference of the
Don’t worry you have never worked with YAML (
title: "Python assignment" tasks: - task-1
codefreak.yml in your assignment root-directory is very slim. It only contains the
title of your assignment and a list of directories that contain the individual
codefreak.yml file in your task directory is where the magic happens. We will define two evaluation steps for our task: A static code analysis with Code Climate (because the config is super easy) and a unit test with
title: Create an addition function in Python (1) description: | Please create an `add(a: int, b: int): int` function in the `main.py` file, that returns the sum of parameter `a` and `b`. Find some useful help in the [official reference](https://docs.python.org/3.7/reference/index.html). hidden: (2) - main_test.py evaluation: - step: codeclimate (3) - step: junit (4) options: image: "python:3.7" (5) project-path: /code (6) results-path: . (7) commands: - pip install pytest (8) - pytest --junitxml=TEST-report.xml main_test.py (9)
|1||The first two lines will add a title and some description to our task. The weird
|3||The first evaluation step is Code Climate. If you don’t add any more options it will perform a basic static code analysis.|
|4||The second evaluation step is the actual unit testing. See the definition docs for a full reference of available options.|
|5||We use the official Python 3.7 Docker image for running the tests. This makes Code FREAK so flexible and allows you to use the Docker Image of your favorite programming language.|
|8||Because the Docker container does not ship with
|9||This is our actual
You can already run the tests locally to check if they behave as expected by invoking the test command
pytest --junitxml=TEST-report.xml main_test.py.
Students can use Code FREAK to program in their browsers. The programming environment is currently based on VSCode. We recommend that you add debugging configuration for VSCode, so your students can execute their source code in the browser. Please follow the official documentation on how to creating debugging/run configurations in VSCode.
The result should be a
.vscode directory with a
launch.json file. Add this directory to you
Before you upload your assignment to Code FREAK you should try to run the tests locally with Docker. After installing Docker you can use the following command-template to test execution locally:
docker run -it --rm \ -w <project-path> \ -v $PWD:<project-path> \ <image> \ sh -c '<command #1> && <command #2> && <command #3> && ...'
<variables> should be replaced by the corresponding values from your task’s
If you are on Windows replace the
This is an example for our
task-1. Execute the following command in the
docker run -it --rm \ -w /code \ -v $PWD:/code \ python:3.7 \ sh -c 'pip install pytest && pytest --junitxml=TEST-report.xml main_test.py'
If your assignment is ready to be uploaded to Code FREAK you have to create a
tar archive. The archive name itself does not have to follow any naming scheme.
The root directory of the archive has to contain the assignment
codefreak.yml file! So do not create a archive that contains another
assignment-123 directory. Be warned that many archive programs do this by default. The safest way is selecting the
codefreak.yml file and all
task-* directory and use your context menu to create an archive of these file.
/my-assignment.zip ├─ codefreak.yml ├─ task-1/ │ ├─ codefreak.yml │ ├─ main.py │ ├─ main_test.py
/my-assignment.zip ├─ my-assignment/ │ ├─ codefreak.yml │ ├─ task-1/ │ │ ├─ codefreak.yml │ │ ├─ main.py │ │ ├─ main_test.py
Code FREAK executes all commands on Linux. There might be cases where the file permissions are important. Either use a
After you created your archive go to your Code FREAK installation and create a new assignment by uploading the archive.