You are here

How to use a Rating sheet

The Rating sheet lets participants express their opinion of the items listed on the sheet.

Compared to traditional ways of rating or prioritizing items such as allocating sticky dots on a paper wall, Rating provides much more time for reflection by the individual user, simply by enabling all participants to rate in parallel rather than one after another. Anonymity enables honesty, digitalization instant aggregation and presentation of results. In short, the Rating workspace produces faster, better reflected and more valid results than traditional procedures ever could.

1 Preparation of the Rating sheet

Preparation of the Rating sheet is fast and easy - assuming you are clear what you want to achieve. If you are not clear, we'd like to think that the very process of configuring the Rating sheet helps you sort the inevitable issues of logic and purpose.

In preparation of a rating, Facilitators must

  1. Specify the criterion and the rating method
  2. Decide on the level of anonymity
  3. Review the rating instruction
  4. Enter (or copy & paste) the items to be rated

1.1 Criterion and Rating method

The criterion is the property you want to rate the items on, for instance, "Importance" because you do not want to waste time on anything that is unimportant, or "Urgency" because first things must come first, or "Effectiveness" because it does not make sense to discuss measures that have no impact on the issue, and so on.

The Rating method is the method by which participants express their ratings. The most popular (and default) rating method is 'numerical scale', 0 to 10 being the default of very many available scales. Beyond numeric(al) scales, MeetingSphere offers all customary rating methods i.e. "Estimate", "Multiple selection", "Rank order" and "Budget allocation".

Most Facilitators specify the criterion and method of a rating directly when they add the Rating to the Agenda via the 'Plus' menu of the Meeting dashboard. Criterion and rating method can be changed in the Basic settings tab of the Rating sheet. Check out that help page on the suitability of different rating methods for specific purposes.

1.2 Anonymity

By default, rating is completely anonymous. You can specify 'teams' i.e. a demography by which ratings are tagged in the Anonymity settings tab.

1.3 Rating instruction

The default rating instruction is computed from the selected rating method and the criterion following the pattern 'Please rate by [criterion]'.

You can change this to something more expressive by editing the Instruction in the Rating sheet workspace. 

1.4 Rate items

You can create rate items 'manually' by posting them via the bottom input box. Often, you will copy and paste rate items from another tool, for examples, ideas from a Brainstorm.

1.4.1 Randomization

Please be aware that the order of rate items is randomized on the rating sheets of participants. Randomization cancels out distortions such as ratings of a given item being influenced by that item sitting near the top or the bottom of the list or directly after another item. In practice this means that there is little point in sequencing items in your view of the rating sheet.

Disable randomization in the View settings tab only for good reason.

1.4.2 Merging of duplicates

If you have duplicate rate items, you may want to eliminate them prior to rating by dragging and dropping the 'duplicate' item on the 'original' item. On confirmation, the duplicate becomes a comment on the 'original' which remains in the list. Eliminate duplicates carefully if you use a rating method such as 'multiple selection' or 'budget allocation' which is sensitive to duplicates or if you have a fixed quorum of items to take forward after rating.

In contrast, rating by 'numerical scale' is indifferent to duplicates: similar ideas will receive similar ratings and sit next to each other in the results table. When looking for the 'top ten' items, you must simply be aware that you may have to go further down the list because, for instance, top-scoring items 3, 4 and 5 as well as 7 and 8 are duplicates and should be counted as 1 idea. In our example, we would not count items 4, 5 and 8 towards the quorum of "ten", take forward 13 items from the results table and boil the down to 10 by merging in the target workspace. 

1.4.3 Folders

You can structure the rating sheet (or subsequently, the results table) by folders.

For this, expand the folder frame, create folders and sort the items into the folders. Switch 'Sort rate items' makes the folders visible on the rating sheet. When folders are visible, randomization of items is limited to the relevant folder.

Note: If you copy items from the Brainstorm workspace to a rating sheet ideas will 'remember' their folder.

In practice, folders on the rating sheet usually add very little. However, they can be quite useful for analysis in the results table and the results chart.

Tip: Use folders in analysis but do not convolute the ballot sheet by displaying folders unless for good reason.

2 The process of rating

The rating process is straight forward

  1. Open the Rating sheet for participants
  2. Make sure the instruction is understood
  3. Let participants rate the items
  4. Submit the rating sheet

Facilitatorss can participate in rating or abstain simply by not submitting their rating sheet.

Rating with MeetingSphere is fast. Do not try to go faster by rushing your participants as you do not want anyone to complain that the results are less than solid as "we didn't have time to think about it".

Unless you have enabled "resubmission" in the Basic settings tab, participants are navigated to the Lobby when they submit their ratings.

Close the Rating sheet workspace before you access the results table or chart.