Creating a Scorecard - Thalamus Holistic Review

Thalamus allows program users to build customized Scorecards to utilize for applicant selection and interviews. Creating a scorecard allows your program to deliver your scoring criteria (through questions and their associated competencies) automatically to application reviewers/interviewers, allowing your interviewers to score applicants directly in Thalamus and Cortex. Programs can create as many scorecards as needed to satisfy all applicant scoring activities for your program and you can customize scorecard weighting. 

Please note: For ease Thalamus Holistic Scoring is prebuilt with Competences, Questions and Scorecards. However customized Competences, Questions and Scorecards can be created for your program.

Competencies & Questions

To build a scorecard, Programs must first create Competencies & Questions. Competencies are your foundation for evaluating applicants at all stages of the recruitment process and across Thalamus products, including Cortex (for selecting applicants to interview) and Thalamus Core (for evaluating applicant interviews).

When creating competencies, think of answers to these questions:

• Who are you looking for?

• What qualities do they possess?

• What are the characteristics of trainees who flourish in your program?

Examples of competencies may include academic performance, medical knowledge, professionalism, communication, voluntarism, geographic connection, and more.

Competencies and questions are then used to build scorecards in the next step, allowing reviewer and interviewers to submit applicant evaluations throughout application review/screening, interviews and/or ranking.

Create a New Competency

1. Navigate to the Score Administration page from your top menu bar.

2. Click the “+ Add Competency” button on the bottom of the “1 Competencies & Questions” page.

3. The New Competency editor module will appear. The pencil and paper icon will appear as teal when editing a new or existing competency. Click the trash can icon to remove the competency or cancel your edits.
   NOTE: A competency may not be deleted if there are questions added to that competency.

4. Click the “New Competency” box and type your competency name.

5. Click the “Competency Description” box and write a short description of the competency.

6. Click the “Save” button to save your new competency.

7. A green notification box will appear in the upper right hand corner of your screen when the competency is saved successfully.

Add Questions to a Competency

1. Click the pencil and paper icon of the competency you wish to add the question to. The competency editor module will appear.

2. Click the “Add Question” button and the New Question module will appear underneath the competency name and competency description.

3. Click the “New Question” box and type your question or statement.

4. Click the “Question type” Dropdown menu to choose which type of question you wish to create. For all question types, the option for Unable to Assess allows the reviewer to select and exclude this question from the score rollup. Additionally, all question types provide the option to enable a note field adjacent to the question.

   • Score offers configurable upper and lower limits setting the allowed range. Reviewers will choose a number within the allowed range.

   • Dropdown supports selection of one response from a predefined list of choices. A custom score value may be assigned to each choice.

   • Levels offer a range of 2-10, a title and description for each level and also supports a response between levels. The configurable scores are not visible to the reviewer. 

   • Likert offers a range of 2-10 with customizable labels. The configurable scores are not visible to the reviewer.

   • Yes/No offers a binary choice. The option for Unable to Assess enables a third, non-binary response when needed.

   • NOTE: Question Tooltip = supports a rubric in the tooltip adjacent each question. This tooltip can contain a couple of paragraphs of text if desired.

5. After choosing the Question type, users may customize the available answers to be selected by reviewers.

6. When all settings are configured for the question, click “Save Question” in the bottom right corner of the new question module.

7. A green notification box will appear in the upper right hand corner of your screen when the question is saved successfully.

Create a Scorecard

After creating competencies and questions, users may build Scorecards.

To build a Scorecard:

1. Click the “2 Scorecards” step at the top of your Score Administration page.

2. Click the “+ Add Scorecard” button on the bottom right side of the page.

3. The Scorecard editor will appear allowing you to customize:

   • Scorecard name

   • Scorecard description

   • Where the Scorecard will be used (in Cortex, Thalamus Core, or Disabled)

   • If the Scorecard should appear first in the list of available scorecards if there are multiple

   • Any question from any competency can be added to any scorecard

4. To add questions to the Scorecard, click the “Manage Questions” button and the “Manage Questions for Selection” module will appear

5. The Manage Questions module will present all available competencies and their associated questions. You may also search for questions in the by typing in the Search bar on the top of the module. For example, typing “communication” will pull all available questions with the word “communication.”

6. Select the box(es) next to the question(s) to include in this Scorecard.

7. The sum/average button on the scorecard configuration controls how the scorecard score is created. The aggregation page controls how the scorecards are aggregated together.

8. After selecting all of your desired questions and the Scorecard Score type, click the “Save Changes” button on the bottom right side of the Manage Questions module.

9. The Scorecard will now appear as an option for reviewers when scoring applicants in Cortex and Thalamus Core.

Manage Reviewer Weighting

You can customize how individual scores are aggregated across your team. By default, scores are weighted equally (100% each) and averaged. Reviewer weighting applies to all program scorecards and score types, including Overall Score.

How to Assign Review Weights:

1. Select "3 Reviewer Weighting" at the top of the screen
2. Click “Add Reviewer”
3. Click in text box labeled “Reviewer” to select assigned reviewer
4. Role will automatically populate once you add a Reviewer
5. Adjust “Weight” if applicable
6. Click “Save Changes”