This page is informational purposes only. It lets you know the default consensus metrics for Likert scale questions, default statistical measure for Group Stability calculation, default value for Stability Threshold measurement, and so on. You can still change all the settings on a question-by-question basis later on in your survey, these settings are just default guidelines.
Consensus Change Alert: will display a popup window in Real-time Delphi surveys to notify panelists when somewhere in the survey on a page other than the current page the panelists is on a consensus for a question has changed (consensus reached or lost), or its stability level has shifted since the last time panelists answered it. This could be a useful feature normally in small surveys that can notify people when they return to their questionnaire that something somewhere has changed. Evaluate the situation that you can use it for as keeping it on could be annoying to panelists in long running surveys as popups will be opening and closing nonstop as others are filling out the survey.
Display Label for Consent: displays text for consent status in all pages and reports. Default is "Consent". You may use any other labels such as "Consensus In", "Agreement", "Consensus", and so on.
Display Label for Dissent: displays text for dissent status in all pages and reports. Default is "Dissent". You may use any other labels such as "Consensus Out", "Disagreement", "Dissensus", and so on.
Display Label for Undecided: displays text for undecided status in all pages and reports. Default is "Undetermined". You may use any other labels such as "Unclear", "Unknown", "Undecided", and so on.
Enable RAND/UCLA Appropriateness Method (RAM): default: No. When enabled, the consensus is calculated using the RAND/UCLA Appropriateness Method (read more or download PDF). All settings under the Consensus tab (except for Consensus Change Alert) are ignored if the RAND/UCLA Appropriateness Method is not enabled. From the Statistics tab, the following statistics can be tracked when the RAM Method is enabled:
IPR: Inter-Percentile Range
IPRAS: IPR Adjusted for Symmetry
IPRCP: Central point of IPR
AI: Assymetry Index
Lp: Lower limit IPR
Up: Upper limit IPR
AGREEMENT: Level of Agreement
APPROPRIATENESS: Level of Appropriateness
NOTE: use only 9-point Likert scale questions for RAM calculation.
Definition of Agreement (RAM): set the criteria for agreement when the RAM method is enabled.
Definition of Disagreement (RAM): set the criteria for disagreement when the RAM method is enabled.
Number/Percentage of Low Extremes to Discard (n): set the parameter for Agreement criteria if the agreement criteria you selected above requires it. It accepts a numerical value or a percent. Use the percentage sign if the value is in percentage (e.g. 8%).
Number/Percentage of High Extremes to Discard (m): set the parameter for Agreement criteria if the agreement criteria you selected above requires it. It accepts a numerical value or a percent. Use the percentage sign if the value is in percentage (e.g. 8%). If value is not provided, the default is 0.
Number/Percentage of Low Extremes to Discard (p): set the parameter for Disagreement criteria if the disagreement criteria you selected above requires it. It accepts a numerical value or a percent. Use the percentage sign if the value is in percentage (e.g. 8%). If value is not provided, the default is 0.
Number/Percentage of High Extremes to Discard (q): set the parameter for Disagreement criteria if the disagreement criteria you selected above requires it. It accepts a numerical value or a percent. Use the percentage sign if the value is in percentage (e.g. 8%). If value is not provided, the default is 0.
Number/Percentage of Ratings that Must Fall in Lowest Range (x): set the parameter for Disagreement criteria if the disagreement criteria you selected above requires it. It accepts a numerical value or a percent. Use the percentage sign if the value is in percentage (e.g. 8%). If value is not provided, the default is 10%.
Number/Percentage of Ratings that Must Fall in Highest Range (y): set the parameter for Disagreement criteria if the disagreement criteria you selected above requires it. It accepts a numerical value or a percent. Use the percentage sign if the value is in percentage (e.g. 8%). If value is not provided, the default is 10%.
Calculating Disagreement (Example):
Calculating Appropriateness:
Appropriateness is determined using the median of the panelists' ratings and the degree of disagreement among the panelists. Its values can be (on a 9-point Likert scale ):
- Appropriate: when the median is greater than or equal to 7, without disagreement.
- Inappropriate: when the median is less than or equal to 3, without disagreement.
- Uncertain: when the median is greater than 3 and less than 7, or in case of disagreement.
- Necessary: when it is appropriate and the median of the necessity is greater than or equal to 7, without disagreement.
Visualization of Appropriateness and its associated statistics in the survey questionnaire:
