Information for Survey Participants
Welcome to the knowledge base for Survey Participants. Please select one of the following sections to review.
Calibrum: the developer of the Delphi platform, Surveylet. Calibrum, Inc. is a US corporation, located in Saint George, Utah, United States.
Surveylet: online survey platform utilizing the Delphi technique. Surveylet is developed by Calibrum.
Survey Response: a panelist's response to the entire survey.
Answer: a panelist's response to a question.
Survey: a logical entity containing a group of pages and questions. A Survey is a template for questionnaires. A survey is a group of questionnaires. The survey is what the survey administrator designs using Surveylet.
Questionnaire: a single panelist's survey data, an instance of the Survey. It is a physical entity which is a representation of the Survey. When a panelist fills out a survey, a new questionnaire is created with a unique ID where all answers for all questions, on all pages made by the panelists are collected under.
Calibrum is fully DPF compliant. For more information on DPF, please visit Data Protection and DPF. For data security and privacy information, visit our online security and privacy pages.
The Delphi method, also known as Delphi technique or Delphi forecasting, is a group decision-making technique used to facilitate a panel consultation and to help making decisions by taking each participant's point of view into account. It is a form of online dialog that seeks consensus and contextual comment over time through responses that can be changed and commented on. Participants can change their responses if appropriate, and the can add comments to ask for clarification or to explain their positions. (Read more)
There are two Delphi methods available:
Real-time Delphi: one of the main Delphi methods allowing real-time consensus-building. If your survey is a Real-time Delphi survey, you will be able to see group analytics and feedback right after you saved your answers.
Multi-round Delphi: a Delphi technique utilizing survey rounds where consensus is built over multiple survey rounds. If your survey is a Multi-time Delphi survey, you will be able to see group analytics and feedback of last survey round for all questions you have answered.
If your administrator sent you a survey link via email, you should be able to enter the survey by clicking on it. It is by default appears as "Take Survey" link as shown below:
The "Take Survey" link is your survey invitation link (also known as Questionnaire URL). It is an encrypted personalized secure link to your very own questionnaire and should not be shared with anyone else.
Notifications
Survey administrator may send you email notifications using Surveylet, in which case the email sender's name will be your survey admin's full name (e.g. John Smith), and the actual email will be "info@calibrum.com". If you do not receive emails for some reason, you will need to white list the following emails: info@calibrum.com, support@calibrum.com. Contact your organization's IT department to whitelist these email addresses, in case they were sent to our junk folder.
Calibrum do not send out unsolicited emails and you may opt out from receiving emails by clicking on the "Unsubscribe" link at the bottom of your Calibrum emails sent by Surveylet (shown below) and send us an email to request removal. We will remove your email address from the survey participants list and delete all your existing responses and collected data for you. This cannot be reversed, so please carefully review if you want to opt out from the survey.
Access Denied
Your access to the survey may be denied if the survey is closed, or if you are no longer the member of the survey participants group (either the administrator removed you, or you opted out).
Access Survey
Depending on how your survey administrator set it up, you may enter the survey directly through the invitation link and can start filling out the questions; or the survey invitation link may take you to the survey registration page prompting you to enter your full name and email address to proceed. Please note that this page may be personalized by your survey admin, so the page below is our default screen:
Invitation Email
When you click on "Let's Start!" - again depending on how your survey admin set it up - you may be taken directly to the survey to fill out the questions, or you may receive a survey invitation email with an encrypted personalized survey link to your survey. This is the message you receive if you are sent the survey invitation link:
Reenter Survey
You may stop filling out your survey at any time and reenter the survey by clicking on the survey link again. Your data is collected and automatically saved when you are navigating forward in your survey. Your data is not lost when you exit the survey; it is stored and will be presented to you when you return.
When you enter the survey you will be presented with one or more pages and on each page one or more questions. An example page is displayed below.
Standard Questions:
Matrix Questions:
Survey Navigation
If the admin enabled it for you, you will be able to navigate directly from one page to another using the page selector drop down at the top of each survey pages. Navigation can also be done through the blue << (Back) and >> (Next) buttons at the bottom of the page. Your answers for all questions on the page are saved by either clicking on the NEXT >> button, or the SAVE button in the right lower corner of the page. Depending on the administrators preferences, the SAVE button may appear right above the blue << and >> page navigation buttons.
Since your answers are saved as soon as you navigate forward, or click on the SAVE button, Surveylet does not have a FINISH button like in other surveys; and you can come back later at any time without worrying about losing your answers you provided in your current browser session.
Please note that using the BACK button or the page selector dropdown will not save your page data; you must use the SAVE button to record your data first otherwise you will lose your answers.
Switching Devices
You may switch devices and maybe start filling out the survey from your mobile phone first and then continue when you arrive home or work using your laptop or work station without any loss of data.
Question Types
Chances are you will need to fill out questions of different types (radio button, checkbox, ranking, scale, matrix and so on). While has a different purpose and may look different, the Question Analytics widget right under the question will be similar. The Analytics widget may contain one or more tabs:
Analytics Tabs
Under each question, you may see the Analytics widget with one or more tabs. The available tabs that may be displayed are as follows:
Standard Questions:
Matrix Questions:
My Comment Tab:
For questions where commenting (the "My Comment" tab) is enabled by the survey administrator, you may (or are required to) enter your reasoning behind your answer you gave for the question. There is no limit to the length of your open end input, fee free to type as much as you wish.
Standard Questions:
Matrix Questions:
Consensus Tab:
Displays information related to group consensus and response counts.
Standard Questions:
Matrix Questions:
Consensus: choice or value of consensus as agreed upon by all participants filling out the question. It is what the team has agreed upon, but it doesn't mean that it received the required level of agreement to be considered as an Agreement. It is determined by the status of consensus which measure is displayed in Consensus Status (shown below). Consensus could be a numeric value for numerical questions, or a question choice for multichoice and ranking questions. The Consensus displays the label associated with the group consensus measure. Consensus may be calculated via different statistical measures configured by your survey moderator; which measure is not displayed for you unless specifically explained to you by the survey moderator. Consensus is displayed on the Consensus tab inside the gauge with green or red background:
Consensus Status: the status of group consensus, can be either Consensus or Disagreement. If there is a clear choice of consensus for the question, and the level of agreement has reached the required threshold set by the survey admin for this question, there is consensus, otherwise disagreement.
Agreement Level: the strength of consensus. Indicates how strongly the group agrees upon. 0-100% , with 0% is the lowest and 100% is the maximum strength.
People Invited: the number of people who have so far entered the survey in current survey round.
Responses Received: the number of people who have so far provided answer to this question in current survey round.
Revisions Made: the number of revisions made by all survey participants to this question. A panelist may revise his or her opinion multiple times; each time it counts as a new revision. Revision occurs if panelist changes his or her selection or comment.
Graph Tab:
This tab displays distribution of answers, representing the percentage of people selecting each question choice. Statistically speaking, each answer provided by panelists for question contains the frequency or count of the occurrences of answers within a particular group or interval, and in this way, the chart summarizes the distribution of values in the sample. Hovering your mouse over the orange vertical bars displays a popup window with details on percentage and response counts the selected choice received.
Standard Questions:
Matrix Questions:
Stats Tab:
Surveylet can display 30 different statistics under the Stats analytics tab such as median, mode, mean, and so on. While explaining and teaching you all the statistical methods used by Surveylet is out of the scope of this introduction, you may want to check out this webpage as a beginners guide tutorial on statistics.
Standard Questions:
Matrix Questions:
Comments Tab:
Comments are all the open text input provided by all participants as reasoning for their answers.
Standard Questions:
Matrix Questions:
When you click on the "Click to view (4) comments" as shown above with by the red right arrow, you are taken to the list of answers given for the selected choice (below). You may also be able to rate other participants' comments, if this feature has been enabled by the survey administrator. In the example below, two rating scales (Importance, Relevance) have been switched on:
Standard Questions:
Matrix Questions:
Stakeholder Analytics:
By default, Surveylet will let you see the consensus analysis for the entire survey group, however, in cases you may be interested in seeing the consensus by different demographics groups. Here's where the Stakeholder Analytics comes into place. If enabled by your survey administrator, you will see a "Click to view consensus by..." hyperlink displayed under your choice list.
Standard Questions:
Matrix Questions:
Clicking on it will display a popup window to display consensus by a cohorts. There are two different ways the stakeholder analysis can be displayed: either via a popup window, or an embedded window.
Popup Window:
Embedded Window:
Each survey page contains a footer section at the bottom of the page. The footer has the following sections: progress bar, navigation buttons and footer option buttons. Depending on how your survey administrator has configured the footer, you may see one of the following theme:
Sticky Footer:
Sticky Footer with the SAVE button Only:
Traditional, Non-Sticky Footer:
Progress Bar:
Displays the percentage of questions you have provided answers for. These may include optional questions also, depending on how your survey administrator set up the survey. Questions you have not yet provided answers for are listed on the "missed questions page available through the "To Do" button in the footer.
Private Chat
A one-on-one private communication channel between you and your survey administrator only. No other survey participants can view the communication happening between you and your moderator. When you receive a private chat, it will also come as an email with a direct encrypted private link to you survey chat window which you can click on to go directly to your survey and view and reply to the message.
Missed Questions:
Displays list of questions that you did not fill out. This include optional questions as well but each question will be flagged with required or not required flag so you know if they missed them or not. In a matrix question, only the the title of the matrix question will be listed and not every single cell you missed.
Exit Survey:
You can quit and exit the survey at any time. Clicking on the Exit button does not mean that you cannot return to the survey later on and edit your answers; you can come back at any time. Your answers will be there so you can continue even after you exited the survey.
Next Button:
The Next button saves all answers for all questions on current page and will take you to the next page.
Save Button
The Save button saves all answers for all questions on current page and keeps you on the current page. With this button, you can answer questions and then see the question analytics for them without having to navigating back again, which would happen when using the Next button.
Back Button
Takes you back to the last page you visited. If you jumped from page 5 to page 9 due to a page branching logic, and you are on page 9 right now, clicking on the Back button will take you back to page 5.
Return to Survey
You may return to the survey at any time you wish by clicking on the survey invitation link you were provided by the survey administrator.
Saving Answers
Your answers are saved immediately when you click on either the blue >> (NEXT) button or the SAVE button at the bottom of your survey pages, therefore Surveylet does not need nor have a Submit Answers button for this specific purpose.
Talk to Survey Moderator
You may contact your survey administrator (moderator) at any time using the "Chat" button at the footer of survey pages. This is a private one-on-one communication window including no other participants, only you and the moderator.
Missed Questions
To view questions you missed, click on the "To Do" button in the footer.
Labeling
Please note that since buttons and labels (e.g. Save button label, To Do button label, My Comment text, etc.) can be configured by your survey administrator, what you actually see displayed may differ from what is explained on this page.
Contacting Survey Moderator
Emails you may receive from Surveylet are sent through info@calibrum.com with your moderator's name as Sender. Instead of replying to emails directly, make sure you contact your moderator as your emails are not monitored and responded to by Calibrum staff.
Receiving Emails from Calibrum
If you are not receiving emails from info@calibrum.com or support@calibrum.com, or if they are sent to your bulk folder or marked as spam, you may need to contact your IT department to whitelist these email addresses. We do not send unsolicited emails. All emails you receive are at the discretion from the survey moderator. For more information on how to unsubscribe and remove yourself from a survey, visit the Opt-out from Survey section on this page.
For any questions related to your survey, please contact your survey administrator directly.
Happy Surveying!
Calibrum