Judging

The Judging Process

SFIAB makes several assumptions about the judging process. These may not match perfectly with every fair, but often, at least part of the judging system can be used to accommodate the needs of the fair. You are free to use, or not use, any of the judging features.

Categories and Divisions and Challenges

Judging assumes projects are divided into age Categories. These are often Junior (grades 7,8), Intermediate (grades 9,10), and Senior (grades 11,12). The categories can be configured through the configuration pages.

SFIAB assumes that ALL projects in an age category are judged together, and there is a single set of medals for each age category. Historically, categories were further sub-divided into divisions, and each division was judged separately. SFIAB no longer supports this model for several reasons:

It is arguably fairer to the students. It means all the projects in a strong division can be acknowledged, and that the only Junior Mathematics and Physics project on the floor isn't guaranteed to get a Gold if it is a weak project.
It eliminates students who try to do a project for a specific division just to win a medal. Every fair has/had weaker and stronger divisions, Health Science typically is a very strong division.
Many projects in different divisions can be judged by judges with the same expertise, so we can group these projects together for judging
Fewer judges are required
Streamlines the awards and award ceremony

When a student registers now, they must select a Detailed Division. There are about 100 detailed divisions, we use the ISEF divisions for this, see ISEF Divisions. The detailed division is used only for matching judges with projects.

When a judge registers, they must select their top three areas of expertise from the same detailed division list as the students. Again, this is used only to match judges with projects.

SFIAB also asks the students to select a Challenge for their project. Challenges mean nothing. They are not used in any way except to group students into floor locations. They have no impact on judging or judging teams.

The Judging Process

SFIAB assumes there are two rounds of judging. In the first round, divisional judging teams score projects based on three metrics: scientific though, creativity and originality, and communication. These metrics are used to compute a score for each project and rank them first to last. In the second round, a number of cusp judging teams refine and re-order the projects around the barrier (cusp) between gold/silver, silver/bronze, bronze/honourable mention, honourable mention/nothing.

The theory is that a clearly strong gold project does not need to be judged again. Similarly, a project that is ranked in the middle of the bronze projects also does not need to be judged again. Only the projects near each cusp need to judged again to ensure the ordering is correct.

SFIAB assumes that each round 1 judging team consist of a number of judges, usually 3, and that they collaborate and agree on a single score for each project they judge. The timeslot scheduler assumes each judge visits the projects independently, of course, for your fair, you are free to not use the timeslots if you prefer your judging teams travel together. Each judging team must visit the same subset of projects however.

SFIAB assumes that round 2 judging teams also consist of a number of judges, usually 6. The judge scheduler tries to assign round 2 judges from a distribution of round 1 judges.

Special awards judges are not assigned to specific timeslots, they are given a schedule of when divisional judges are NOT at the projects they must visit.

Judge Scheduler

The judge scheduler performs the following tasks:

Deletes all automatically created judging teams (e.g., from a previous run of this scheduler), manually created judging teams are not touched.
Creates new judging teams for divisional, CUSP, and every special award marked as "schedule judges"
Assigns judges to judging teams
Assigns projects to divisional teams and special awards teams
Runs the timeslot scheduler to create a judging schedule for each judging team and project (printable on the reports page)

Timeslot Scheduler

The judging timetable is broken into a number of equal-length timeslots. For example, a three hour judging period may have nine timeslots of 20 minutes each. The timeslot scheduler is run automatically after the Judge Scheduler, or it can be run separately. Some situations where it is useful to re-calculate timeslots without re-calculating judging teams include:

A project becomes unavailable after assignments are made, e.g., they forget to inform the fair they must leave early to catch a ferry until a week before the fair.
A change in the judging length, or the length of the timslots

The timeslot scheduler does not change judging teams, or the projects assigned to judging teams. It only calculates a timetable for each judging team and project. It performs the following actions:

Deletes ALL existing judge timeslot assignments and project timeslot assignments
Computes a timetable for all projects with three different timeslots for:
- divisional judges
- special awards judges
- free time
Assigns all Round 1 judges on round 1 divisional judging teams to a specific timeslot to visit each project. These will match up with divisional timeslots assigned to each project.

NO special awards teams are assigned to timeslots. Instead, the system can print a report for a special awards judging team that lists ALL the special award timeslots for each project. (Basically, when each student will be at their project and won't have divisional judges).

NO round 2 divisional (cusp) teams are assigned to timeslots for three reasons:

we don't know which projects need to be judged,
there are more than 3 judges/cusp team, there aren't enough timeslots for 1:1 timeslot assignments,
There's no guarantee that the divisional timeslots assigned to each project actually allow an even distribution of judges.

Instead, after projects are assigned to these teams (there's a button to do this from the judge score entry), the system will print cusp judging team timeslot reports for each team that lists when each project is available for divisional judging.