You are here

Analysis of results in the Multi-criteria Results table

Multi-criteria Analysis

To rate a list of items on multiple criteria, Facilitators simply

  1. highlight an existing Rating (sheet) in the Agenda
  2. add another Rating (sheet) via the 'Plus' menu of the toolbar
  3. confirm that they wish to extend the existing Rating

Extension of a Rating on one criterion by another Rating on another criterion creates a "Multi-criteria analysis": In addition to the new Rating sheet and results table, MeetingSphere automatically adds a Multi-criteria results table and a Multi-criteria results chart.

Rating on multiple criteria works just like multiple independent ratings, EXCEPT that in Multi-criteria analysis

  • The list of items is identical on all rating sheets (changes apply to all)
  • Anonymity settings run across all rating sheets (changes apply to all)

Moreover, you should stick to one Rating method if possible: It is much easier to analyze results across criteria if they are in the same 'space'.

Also, limit the number of items on which you run a full-fledged Multi-criteria analysis. If you have a long list of items, run a 'cut-off' vote on a criterion items must meet to qualify for in-depth analysis. For example, when looking for measures, rate them on 'effectiveness'. Then analyze the 10 (or 20) 'most effective' measures on their specific 'solving power' regarding 'Problem A', 'Problem B' and 'Problem C'. Add other relevant criteria such as 'feasibility' and '(political) palatability' or whatever may interest you.

    Use of the Multi-criteria results table

    The Multi-criteria table displays the results for all criteria side-by-side.

    • For a summary score across all criteria, display column 'All criteria'
    • For a summary value for each criterion, display row 'Mean of column'

    Columns can be hidden or displayed with a switch in the column header when you open 'Table settings' with the cogwheel  toolbar button.

    To see quickly which items score highest/lowest for any criterion, click on the relevant column header.

    If you have voted 'by team', use the team comparison widget of the bottom status bar to see how teams are aligned or differ in their assessments.

    Highlight 'High Standard Deviation' to spot controversy. If specific items are controversial regarding multiple criteria, this may indicate a lack of (shared) understanding of that item.

    If your table is inherently complex, reduce complexity initially by hiding columns, then reveal more data as you dig deeper.

    Once you are familiar with the basics, explore advanced features like

    • Color coding
      which lets you analyze patterns based on and sensitivity to certain thresholds
    • Calculated columns
      which let you calculate indicator values such as 'risk' from, for example, assessments of 'impact' and 'probability'.

    Accessibility of the table

    Since you and your group may want to analyze some results of your multi-criteria analysis while some ratings are still in progress, Multi-criteria tables are not blocked by a progress meter. Columns

    • whose single results table or chart would be barred by a 'progress meter'
    • are calculated on the basis of such a column

    will simply show a label, 'in progress'.

    To access the Results for that criterion, simply 'close' the Rating sheet.

    Note that if you have good cause to enable access to the results while rating is in progress, you can do so by selecting the relevant option in the 'View' settings tab of the Rating sheet.

    Decision making

    If you are trying to reach a decision in the group, Multi-criteria analysis will often provide a robust basis for that decision, especially, if you have run your analysis after a Discussion on the relevant criteria.

    However, the table will not make the decision for you. Use it as a basis for reaching a conclusion. That conclusion will depend on whether you are looking for a 'broad' solution i.e. something that scores well across all criteria, or wether one criterion is clearly more important than the others which merely need to be 'satisfied' (good enough), not optimized.

    Multi-criteria analysis is, of course, an excellent way of presenting the opinion of the group to decision makers outside the 'room'.