The Code Evaluation Tool is a mechanism for constructing a checklist of requirements for an assignment and running that list automatically against projects in quick succession. This allows teachers to build a checklist, then easily evaluate any number of projects based on that checklist. Projects are evaluated one at a time, but with auto-run enabled, you can update the loaded project by providing a new share link, at which point the rules will automatically be re-run on the new project.
Create a new checklist using the New Checklist card. If there is already an “in progress” checklist, a warning will appear asking if it is okay to overwrite it.
The checklist is given a name.
One or more criteria are added from the catalog using the Add Criteria button.
Some criteria (like [block] used [count] times) can be added multiple times, others (like Read a GPIO pin can only be added once).
Parameters for the criteria item are filled in for a criteria item.
Here a block is selected and used 3 times:
Parameter options are displayed and then selected.
You can also have an Ask AI question as a criteria item in the checklist. You are limited to up to 5 Ask AI questions per checklist.
A criteria item is removed using the trash button.
Load a project into the project view by pasting in a share link or share ID.
The project will load in read-only mode with the project title appearing at the top of the project view.
With a project loaded, the checklist can run. The results are shown after clicking the Run button.
The results view lists each criteria with its outcome.
Note: the Run Button is disabled without loaded project.
Feedback and notes are added using the Add Notes button. The feedback box should resize to fit its content as notes are added. The original feedback remains even if you re-run the rules using the Run button.
An outcome is edited using the provided dropdown.
The new selected outcome.
Auto-run is toggled either on or off using the button in the menu.
If auto-run is disabled, a result’s outcome (i.e. “Looks good”, “Needs work”, etc…) is set to “Not started” automatically if any of any of the following conditions are met:
If auto-run is enabled, any rules that enter the “Not started” state due to the conditions listed above are immediately and automatically re-run with their results updated.
There are pre-built checklists are available on the welcome page. If a selected checklist is already in-progress, an overwrite confirmation prompt is given.
A checklist is exported using the vertical “…” menu near the “auto-run” button. Only the checklist is exported, not the results. For a copy of the results, use the print function in the Results view.
This will download a json file for the checklist.
User can import a checklist from a file using the same “…” menu, or from the card on the welcome page.
Checklist file is selected using “Browse” or dropped directly into the popup. An overwrite confirmation prompted if there is currently an in-progress checklist.
If the page is refreshed (or if the browser closes/re-opens), the current checklist preserved.
Use the print button to create a version of the results with the outcomes and feedback visible (the other UI elements are hidden).
The checklist-view/project-view splitter can be resized. It can also be reset to 50/50 split with double-click.
Slide the splitter to widen the view of the criteria and results.
