The Science Behind Our Student Engagement Score & How It Transforms Team Effectiveness

Students often experience advising as a series of disconnected moments: selecting courses one term, exploring careers much later, and seeking support only when challenges arise. Critical information about their progress, engagement, and goals is often spread across various departments, limiting the ability to intervene early or guide them holistically.

This fragmentation creates a structural gap. Students are left to navigate complex decisions without a clear, unified pathway, and institutions lack the visibility needed to consistently move students toward forward.

A case management approach reframes student success as an actively managed lifecycle rather than a set of transactions. Each student is supported through a structured pathway that integrates academic and career advising.

This model introduces continuity into the student experience. Academic plans are no longer static documents but living frameworks tied to long-term goals. Progress is monitored against defined milestones, and engagement is driven by timely, data-informed interactions rather than periodic or ad hoc check-ins.

Benefits of Implementing a Student Engagement Score

The Engagement Score in Advisor AI reflects how actively a student participates across high quality academic, career, and co-curricular tasks, both within the platform and through self reported student data across their milestones.

“An Engagement Score is simply an indicator of the extent to which a student is investing time in their education - and the research is clear that "Time on Task" is one of the main predictors of student success.” — Dr. Jeff Doyle

This score enables enrollment advisors, academic advisors, career services, faculty and student success teams to:

• Identify students who may be disengaged and require enhanced support
• Identify those moderately or highly engaged and provide them reminders
• Provides real-time insight into student engagement and momentum
• Easily identify and track custom cohorts based on engagement levels and common student attributes (major, class year, custom email lists, etc.)
• Accelerate enrollment, completion and workforce readiness initiatives with data driven outreach

How the Engagement Score Works

The Engagement Score is a single value from 0 to 100 representing balanced, consistent student interaction across the platform. Higher scores reflect stronger momentum toward academic and career readiness goals, while a lower score indicates new users and the opportunity for upward mobility and momentum.

Features Monitored

Drawing on 3 years of  field research and implementation case studies with thousands of students and advising teams members, we have identified the following core features as essential to high-quality advising best practices:

• Explore
• Engagement across exploration (Number. of career assessments taken and program recommendations)
• Engagement across chatbot inquiries (Number. of questions related to majors, academic policies, career pathways, internship preparation, and campus resources)
• Prepare
• Engagement across plans (Milestones completed, number of plans generated, milestones/ notes updated)
• Connect
• Sharing of plans (Tracking if plans are shared with advisors for feedback, downloaded from the system)
• Engagement across mentorship and connections (number. of times links to book an appointment, or in app scheduling, or outreach connections are made to meet with a human advisor)

Weights and Scoring Logic

Different actions across the student lifecycle contribute differently to the Engagement Score. Higher-impact activities that require reflection or interaction (e.g., taking an assessment, requesting feedback from advisors) carry more weight than passive actions like logging into the system. If a feature is not enabled (e.g., AI Assistant), its activity does not influence the overall score.

If the student is a prospective student, the score helps indicate their level of likelihood to enroll in the program (enrollment momentum). If the student is enrolled, and the system is calibrated to academic policies and degree information data (the score indicates momentum towards degree completion and academic momentum), and if calibrated to career services information (career momentum), it indicates workforce readiness at the individual level. This automated mechanism helps teams to provide support in a scalable manner.

Example Scenarios

• Completing a career quiz or advancing on a plan: moderate increase
• Logging in regularly: maintains and stabilizes the score

Diminishing Returns (Balanced Engagement)

To promote well-rounded student activity, the system applies diminishing returns:

• Initial actions in a category provide strong boosts
• Additional repetitions provide smaller boosts
• Prevents over-reliance on a single feature (e.g., taking 10+ quizzes)

This encourages meaningful, broad engagement across development tasks.

Process Summary

1. Student activity is tracked throughout the day
2. Scores refresh automatically at 1:00 AM PST daily
3. Updated scores are saved to the student profile and cohort management tabs within the administrative view
4. Scoring rules are reviewed quarterly
5. (Optional, but recommended to teams) The best way to test and enhance the accuracy of this approach is to compare the results of the engagement data with the actual retention and enrollment of students each term.

The scoring framework is reviewed annually by Advisor AI’s student success expert team to align with evolving features and high quality advising research.

Appendix:

1. Score Levels & Color Indicators

2. Platform View (Administrative view of student management tab with engagement score)