You are here

Overview of Rating

MeetingSphere Rating serves for assessing a list of items by specific criteria. Rating consists of 3 interrelated workspaces:

Rating sheet
which serves for making the assessment by a specific rating method

Results table
which aggregates the individual ratings and presents them for analysis

Results chart
which presents the data of a results table graphically

Multi-criteria analysis. By adding further criteria to an existing Rating sheet, you can set up a 'Multi-criteria analysis' which may utilize multiple specific Rating sheets (one per criterion) or a more complex Multi-criteria rating sheet, which asks participants to rate each item on all criteria simultaneously. The Multi-criteria results table and the Multi-criteria results chart facilitate results analysis across criteria.

How to run a Rating (Overview)

To rate a list of items on, for example, "Importance" or "Urgency", Facilitators add a Rating via the 'plus' button of the Meeting dashboard. This adds a set of three workspaces to the agenda:

  1. The Rating sheet
  2. The corresponding Results table
  3. The Results chart for that Results table

Rating sheet, table and chart are designed as separate workspaces rather than one more complex workspace for easier configuration and greater flexibility in the meeting: Depending on the purpose of the meeting, Leaders can choose to open or close the table and the chart one after the other or in parallel or not at all. 

Preparation

Initial configuration of the Rating sheet usually occurs in the Meeting dashboard where Leaders select the

  1. Rating method
    for example, numerical scale 0 - 10
  2. Criterion
    by which the rating shall occur, for example, 'Importance', 'Effectiveness' or 'Urgency'

This initial configuration of the Rating sheet in the dashboard

  • automatically generates the Rating instruction, for example, 'Please rate the items by Importance'.
  • pre-configures the corresponding Results table and chart

Leaders can refine the initial configuration by entering the workspace and opening the settings  from its toolbar.

The Rate items, i.e. the ideas, opinions, facts, options or whatever shall be rated, can be added in advance or, if they are generated by a previous step of the same meeting, copied from that workspace just in time.

Rating

The process of Rating is simple and - most certainly to the participants - self explaining:

  1. The Leader opens the Rating sheet for participants
  2. Participants go through the list and rate the items as instructed
  3. Participants post the Rating sheet which auto-navigates them to the Lobby
  4. The Leader closes the Rating sheet

before reviewing the results by table or chart.

Results

It is good practice to postpone results analysis until after the Rating is closed. Which is why MeetingSphere, on entering the results table or chart while rating is still in progress, presents a progress meter rather than the incoming results. You can change this default behavior in 'View' settings tab of the Rating sheet if your purpose requires.

Results analysis usually asks 3 questions

  1. Which items have scored very highly (the 'Top X')
  2. Which items do not warrant further consideration?
  3. Where do we disagree?

For this, Leaders either open the Results table or chart for participants or share their screen.

Rating on multiple criteria

While rating by one criterion is excellent for creating a shortlist of items, such a one-dimensional view is often not good enough for informing a decision. For instance, measures which are rated as 'highly effective' must also be 'feasible' and should ideally fit your 'culture' (the way you do things).

In this case, you'd want to rate your items on 'Effectiveness', 'Feasibility' and 'Cultural fit'.

Or, if you are looking for broad solutions which solve several issues A, B and C, you may want to rate your options by their (positive) effect on "Issue A", "Issue B", "Issue C" and add other relevant criteria such as 'feasibility' and '(political) palatability' or whatever may interest you.

Technically, Facilitators have the choice of letting participants apply the different criteria

  1. sequentially, i.e. with one rating sheet per criterion, or
  2. in one 'shot', i.e. by setting up a Multi-criteria rating sheet. 

Sequential rating combines the benefits of greater discriminatory power with lower cognitive load: While working through the first Rating sheet, participants only need to think of that criterion, e.g. 'Importance', then turn to the next rating sheet and apply that criterion, e.g. 'Urgency' to all items, and so on for further criteria. Sequential rating is recommended in most circumstances.

One-shot multi-criteria rating means that the Rating sheet asks participants to rate each item on all criteria, e.g. 'Importance', 'Urgency' and 'Feasibility' before progressing to the next item. This can be convenient but requires higher levels of concentration. You should also be aware that applying multiple criteria simultaneously may produce unconscious and conscious (political) cross-over between criteria.

Note: You want to limit Multi-criteria analysis to the items that warrant this level of attention. If necessary, run a cut-off vote before really drilling into the likely candidates!