Evaluations (pub. 4/9/2024)

Curate an evaluation template to capture feedback, automate notifications to your professional and analyze responses through our evaluation reports.

Evaluation Workflow

Build an Evaluation Template

Associate an Evaluation Template to a Session

Send an Evaluation Template

Report on Evaluation Responses

Timeline for the support of the Version 1 Evaluation framework

 

Published: 4/9/2024.

Evaluation Workflow

The lifecycle of an LCvista evaluation is based upon three major events:

1. creating an evaluation template through the LCvista wizard
2. associating the template to an LCvista session
3. distributing the evaluation to a professional by email

All three steps are required to successfully send an evaluation to a professional, in LCvista. This article will outline the step by step process to address each requirement and include best practices for making the most of this feature set.

Build an Evaluation Template

To build an evaluation template, you must have access to the Evaluations menu, either through the Default Administrator or Evaluations admin role. 

The Evaluations menu provides the following information:

• visibility to the evaluation templates which already exist; their name, unique ID and status
• exposure to the number of sessions that particular template is associated to
• access to further actions that can be taken on an evaluation template (such as delete, preview)
• access to create a new evaluation template

To create a new evaluation template, select +Add Evaluation Template from the top right. Selecting this button will navigate you into the evaluation edit form. Take note of the following:

• the name of the template exists on the top left and can be modified, the template name defaults to Untitled Evaluation
• use the breadcrumb for < Templates to return back to the Evaluations menu
• select +Add Question to begin adding questions to your template (a minimum of one question is required in order to save)
• specify your question type under Select question type; your options include the following (more detail on each later on in this section).
  • Open Ended
  • Multiple Choice
  • Rating Scale
  • Statement
• select Add on the bottom left to add additional questions to your template
• select Required within each question card, to require a participant to answer the question in order to submit evaluation
• select Ask this question for each instructor within each question card, to repeat this particular question for each instructor associated to your session
• Duplicate a question card to create an additional instance of the question. Each question label must be unique, adjust the duplicate label as required.
• Delete a question card to remove the question from your evaluation template; don't worry you will be prompted to confirm your choice before the question is removed!
• select Save on the top right to process your changes.
• after the initial saving of your evaluation template, you will have access to Add  a Logo. This action is visible from the drop down action contained within the name of your template
• Preview is available both from the template name drop down and after selecting Save, to visualize what your professionals will see when responding to your evaluation. Note: the preview contains test data and will assume two test instructors for every question marked as repeat per instructor. 

 

Question Types

Open Ended

This question type allows an administrator to pose a question to which a participant can leave an open-ended response. Use this question type to get suggestions from a participant on how to improve the next event or ideas for the next meeting's hot topic!

Multiple Choice

When creating a multiple choice question, ensure you have created at least two potential responses. Note that you can adjust the display order of the answers you have defined. Use this question type when you are looking for feedback on a specific set of possible responses, such as, "which agenda item was of most interest?", or "which topic required more discussion for next time?"

Rating Scale

Administrators can create a single rating scale and associate to a number of questions, as well as create multiple rating scales tied to a selection of different question sets. For example, perhaps you have a collection of questions where you're trying to understand how likely a participant is to recommend an instructor, or recommend the session to a colleague. In this instance you may wish to set up a rating scale including Unlikely to Very Likely.

At the same time, you may have a collection of questions where you are looking for a numerical response, "On a scale of 1-5 how effective was this instructor in showcasing the advantages of X?" In this instance you may wish to have a rating scale defined of 1 to 5 with a collection of relevant questions associated.

Statement

Finally, our last question type, is not truly a question. Instead, the Statement question type provides a place for administrators to add contextual help to the evaluation form. Is there detail you wish to provide to your professionals on the onset of providing feedback? Perhaps you wish to break up the evaluation by providing context to the second half of the form and your interest in understanding how next year's event could/should be improved.

Or, perhaps you wish to provide a thank you and next steps at the end of your evaluation. The Statement question type can be utilized in each of these ways. 

Evaluation Template Statuses

Draft

When in Draft state, your evaluation template can be modified. While in draft, you will not be able to associate your template to any sessions- either through the session form or through Bulk Uploads. To perform the latter, you must first Publish.

Published

To publish an evaluation template, access the Preview mode of your template and select Publish. Note, you will be prompted to confirm your choice to publish and will have the option to cancel if needed. It is very important to highlight that a Published template can no longer be modified in any capacity. Once a template has been published, it can be associated to sessions, both individually from the session form as well as in bulk, through the Bulk Upload: Sessions workflow. In the event there are additional changes to be made to a published template, you have access to duplicate the form, make the appropriate changes and associate that new form to your session(s).

Disabled

You will have access to disable a published evaluation template. This action will not be available for templates in draft state. A disabled template can remain associated to a pre-existing session, but will not be visible in the Evaluation Template drop down to associate to new sessions or notifications. Use the disable option in those situations where you are looking to reduce the number of template options visible on your session and notification forms.

Version 1

Evaluation templates created in the historical framework will continue to be visible in LCvista. Any existing evaluation templates will be visible from the Evaluation menu, and all will be prepended with 'V1' for easy identification and differentiation to new version 2 templates. Version 1 templates will remain visible in the Evaluation menu for preview and deletion, but new templates cannot be created in the old version 1 design. For more information on access to historical version 1 templates, please review our FAQ article, here.

Associate an Evaluation Template to a Session

Associate Individually via the Session form

There are two paths to associating an evaluation template to a session.

The first is individually, by assigning the specific evaluation template to the session through the session form.

To associate an evaluation template to a session, locate your program from the Manage Training menu. Access the session in question from either the row action menu located to the right of any program on the Manage Training page, or, from the program's preview page.

Within the session add or edit form, scroll to the Evaluation Template drop down. This drop down will expose all active evaluation templates that can be tied to a session and distributed to participants.

Associate in Bulk via the Bulk Upload: Sessions workflow

The second path is to associate an evaluation template to multiple sessions at once, through the Bulk Upload: Sessions workflow.

To process this action in bulk, access the Bulk Uploads menu and select the Add New button. From this page, select Sessions from the Upload Type drop down and download the available template. Use this template to document the sessions you wish to modify, along with the required fields to add or update. Include the unique ID for the evaluation template you wish to associate to your sessions; upload your completed template and process your changes!

Please note, it can also be useful to pull a Session Info report, including the column for Evaluation Template to identify which sessions have an existing evaluation template associated and those that are in need of one! This data can than be added to the Bulk Upload Sessions template to make the appropriate changes all at once. For more information on reports, please visit Reports. For more information on Bulk Uploads, please visit Bulk Uploads: Sessions.

 

Send an Evaluation Template

After creating your evaluation template and associating it to the relevant sessions, the next step is to distribute to your professionals. To fulfill this next step, you will need to configure a Send Evaluation notification.

To create this notification, access the Notifications menu. Elect to add a new notification and specify the Event type as Send Evaluation. After making this selection, fill out the required criteria on the left of the form, including Name (visible to your team of administrators only), Email Subject and Message (both of which are visible to your professionals).

The next step is to decide how this notification should be triggered to your professionals. The options include:

• Manual: distribute manually through the Manage Attendees menu. From Manage Attendees, an administrator can intentionally choose to send evaluation notification to any participant that has a session status in the event. At this time an administrator can also elect to edit the notification before sending. For more on this page, visit Manage Attendees.
• Automatic: distribute automatically based on a selection of criteria defined by your team. To send this notification automatically, select the setting on this template Send Emails Automatically. Checking this box will expose all criteria available to determine when this notification should be sent. The filtering criteria is as follows:
  • Trigger: should this notification be sent to professionals upon completion? At the scheduled end time of the event? Or perhaps a specified time before or after the scheduled end time of the event? Scheduling to send upon completion will trigger the notification at the time a professional's status changes to Passed. Scheduling to send before, at, or after the scheduled end time will trigger the notification based on the session end time specified. 
  • Recipient: for notifications set to send on completion, the recipient will be preset to passed statuses. For all other trigger options, the recipient list can be defined by one or more statuses, including Enrolled, Incomplete, Failed and Passed.
  • Additional Filters: would you like this notification to be sent for a specific evaluation template? Or should it be sent for all evaluation templates? 

 

Report on Evaluation Responses

Once you've sent off your notifications carrying access to your evaluations, the next step is to review and analyze the feedback provided by your professionals. There are three reports available to review the feedback sent in by participants. For each, you will need to first specify the evaluation template you wish to report on.

Evaluation User Responses

This report will return all responses submitted by a participant. This report can include columns for the following:

• User data: including first/last name, any custom fields such as office or department, etc.
• Program/session data: including name of program, program/session ID, delivery method, session start/end datetimes, venue, etc.
• Evaluation Questions: the questions tied to your template. The format for each question label will be "Question Type: Question" for example: "Rating Scale: How likely are you to recommend this course". In the case of a question repeated per instructor, the label will be appended with "-instructor".

This report returns a row for every user response, including the relevant user, program and session data along with the responses submitted by that particular user. Use this report to review all responses submitted by a user for a particular session evaluation.

Evaluation Instructor Feedback 

This report will return the feedback submitted for each instructor associated to a session. This report includes columns for the following:

• User data: including first/last name, any custom fields such as office or department, etc.
• Program/session data: including name of program, program/session ID, delivery method, session start/end datetimes, venue, etc.
• Instructor Evaluation Questions: a column for each question which was marked as repeat per instructor. The format for each question label will be "Question Type: Question -instructor" for example: "Rating Scale: How likely are you to recommend this course". 
• Evaluation Questions: a column for each evaluation question, not specified as one that should be repeated for each instructor. The format for each question label will be "Question Type: Question" for example: "Rating Scale: How likely are you to recommend this course". 

This report will return a row per user response, returning the feedback submitted to each question. The row for each user's response will be repeated for every instructor associated to the session, where the output for each instructor question will reflect the user's response to that question by instructor. For example, if a session has instructor Jane Doe and John Smith, this report will return 2 rows for every participant; one row representing participant's feedback for Jane Doe and a subsequent row representing the feedback for John Smith.

Use this report to evaluate the responses coming in for all instructors tied to a particular session, or for an instructor across multiple sessions which share the evaluation template.

Evaluation Aggregate Data

This report aggregates responses for a session evaluation and exposes how many evaluations in total were sent and responded to. This report includes columns for the following:

• Program/session data: including name of program, program/session ID, delivery method, session start/end datetimes, venue, etc.
• Total Evaluations sent: the number of unique evaluations sent for a particular session.
• Total Evaluation Responses Submitted: the number of evaluations submitted for a particular session. 
• Evaluation responses count: the total number of users who responded with this answer to the question.
• Evaluation responses percentage: the percentage of users who responded with this answer to the question.
• Evaluation instructor responses count: the total number of users who responded with this answer to the question, per instructor.
• Evaluation instructor responses percentage: the percentage of users who responded with this answer to the question, per instructor.
  
  

This report will allow you to identify which sessions are awaiting evaluations from participants and can expose how participants are responding to each question.  Does the data show that the majority of participants rate the effectiveness of the session at a 5, on a 1-5 rating scale? Are the majority of participants Very Likely to recommend the session to a colleague? 

Use this report to identify how many responses have been received overall per session, and how the majority of your participants are answering your evaluation questions.

 

Timeline for the support of the Version 1 Evaluation framework

The historical version 1 design for evaluations will continue to be supported through the end of July 2024. During this time, version 1 templates can be previewed, associated to sessions, notifications and reported on. 

While version 1 evaluation templates will be supported during this time period, it is recommended that firms build all of their evaluation templates in the new version 2 framework. 

After July 2024, professionals will be unable to access version 1 evaluations templates. Any outdated notifications sent to a professional will navigate the professional to a page stating the form is no longer valid. For more information on what to expect during this temporary period, and impact, please visit our FAQ article, here.