Design Research in Online Learning Environments

Georgia Institute of Technology

Team
Solo
Role
Designer
Duration
12 Weeks
Focus
Design Research, Need-finding

Overview

A solo design research project at the Georgia Institute of Technology. My aim is to use human-centered design methods to find the desires, needs, and frustrations of students within an online-learning environment and provide recommendations for design solutions.
Introduction
Discovery
Define
Test
Outcome

Project Opportunity

Explore the potential discrepancy in students’ experiences in an remote and in-person of the same class. As a teaching assistant for the discussed course, I aimed to conduct primary and secondary research to uncover the state of students’ experiences and inform the development of next steps.

Project Impact

I validated that students of the online and in-person course have different learning experiences. In turn, I defined the components of a successful learning community and identified actionable next steps for the online course to take in order to emulate a successful community.

Project Background

It all started when I became a teaching assistant....

CS 1301 is Georgia Tech’s introductory computer science course. The class serves mainly first time GT students and first time programmers, and there is both an in-person and online version of CS 1301. Personally, I was a student in the in-person class and then became a TA for the online section of the class. The online version of CS 1301 takes a distributed learning approach within a MOOC (massive open online course) structure. It aims to touch on the benefits of in-person learning with asynchronous and remote structure. While I was helping teach the online section, I hypothesized that students in the online section had different experiences than I had during the in-person course.

My goal was to discover insights that may inform the development of new human-centered service offerings that could be integrated into future courses.

Design Scope

This is a needfinding project

This project seeks to define the major components of a successful learning community and provide suggestions to how the online course can better provide the benefits of in-person community learning while still maintaining the standards of a MOOC (massive open online course).

My Objectives

To analyze, assess, and explore

A.

Analyze observations to discover insights that may inform the development of new human-centered product or service offerings that could be integrated into future virtual-learning settings.

B.

Assess the application of human-centered design methodologies, ethnographic-needfinding, and design research techniques in a real-world setting.

C.

Explore the potential utility design-thinking can provide when applied in a non-commercial/ educational space.

Literature Review

Creating a research baseline

To gather initial data for this project, I read an overview of design research methods and how design plays into learning. I also read The Distributed Classroom co-written by Dr. David Joyner to learn more about the current structure of CS 1301, and visited the pre-course survey taken by current online 1301 students about their initial motivations and thoughts about the course.

Just Enough Research
Erika Hall
Design for How People Learn
Julie Dirksen
The Distributed Classroom
David A. Joyner and Charles Isabell
Pre-Course Survey
CS 1301 Class

Mindmapping

Exploring potential themes and relationships

With this initial data in mind, I began to mind map, and came up with three common themes.

Attempts of replicating in-person course benefits online
There are potential (long + short term) benefits working on topics with other students
Students choose this class to work without the constrains of in-person classes

KWH Framework

How do I move forward?

Using this card-sorting framework, I identified what I knew, and what questions I had to determine how I should proceed.

What Do I Know

  • Majority of students have not attended help desk or recitation
  • Scheduling conflicts lower participation in synchronous resources
  • Certain students have expressed interest in collaborating with others, but as of now, do not
  • Certain students have expressed interests in a more personalized learning experience
  • There are a variety of non-lecture resources used by students to learn and engage with course material

What ?s

  • How do students currently use and interact with course resources?
  • What are the long-term benefits of course resources?
  • How have past students benefitted from this course?
  • Would integrated opportunities for collaboration be well received?

How Do I Proceed

  • Read literature regarding established themes
  • Survey current online 1301 students
  • One on one interview with past and current students

Logic Model

Defining elements of a learning community

From my thoughts on mind-mapping and KWH, I found three prominent themes of learning: collaboration/relationship building, attendance, use of course resources.

Relationship Building
Attendance
Use of Course Offerings

Literature Review

Exploring elements

The next step in my process was to conduct a more specified literature review, looking at articles which helped me further define each theme. The bolded words are key words I used to search up credible articles. After a vetting process, I selected 10 to read in depth.

Collaboration, Learning Community, Group
  • Supporting Online Group Projects
    Sung Youl Park
  • Motivating Online Collaborative Learning: Design Implications from a Learning Goal Orientation Perspective
    Hae-Deok Song
  • Building a Sense of Community in an Online Class
    Beverly Davenport and Doug Henry
  • The Effect of Online Collaboration on Adolescent Sense of Community in Eighth-Grade Physical Science
    Jillian L. Wendt and Amanda J Rockinson
Attendance, Classroom Retention, Performance
  • An Analysis of the Technology Acceptance Model in Understanding University Students Behavioral Intention to Use e-Learning Sung Youl Park
    Sung Youl Park
  • Do Attendance Policies Improve Student Performance? The Relationship among Attendance, Class Policies, and Grades
    Matthew Rendleman
Online Course Resources, Learning Styles in Classrooms
  • A Study of Student’s Perceptions in a Blended Learning Environment Based on Different Learning Styles
    Buket Akkoyunlu and Meryem Yilmaz Soylu
  • Factors Affecting Student Attitudes toward Flexible Online Learning in Management Education
    Judy Drennan, Jessica Kennedy and Anne Pisarki
  • Building a Model Explaining the Social Nature of Online Learning
    I-Chun Tsai, Bosung Kim, Pei-Ju Liu, Sean P/ Goggins, Christiana Kumalasari, and James M. Laffey
  • Online Learning: Social Interaction and the Creation of a Sense of Community
    Joanne M McInnerney and Tim S Roberts

Card Sorting

What themes constitute each element?

I used card sorting methods in order to sort and analyze information from my literature analysis.

Element Themes

Each element has sub-themes

After much reading, I recognized that articles that spoke to each element would reference these themes as well.

Class Policy

Correlation to Success
Personal Growth

Academic Growth

Course Satisfaction
Variation of Learning Styles

Difficulties of Online Learning

Revision

Emending element definitions

After card sorting, I was emended the 2 of the 3 elements to better convey their role in the overall learning community.

Attendance
Attendance in Synchronous Resources
Collaboration / Relationship Building
Use of Course Resources
Support for Different Learning Styles

Complete Ecosystem

A functional learning community consists of...

All of these three elements are necessary to create a sense of learning community.

Collaboration / Relationship Building
Attendance in Synchronous Resources
Sense of Learning Community
Support for Different Learning Styles
=

Logic Model Iteration

How does each element relate to one another?

I began to relationship map in order to see how exactly these three elements related to one another and contributed to an overall sense of community.

Final Logic Model

Relationship of elements

This logic model shows the relationship between the three previously defined elements and how they create a sense of learning community.

Survey

Validating the proposed logic model

I constructed and sent a survey to current CS 1301 students in the online section. It received 89 responses. There were both qualitative and quantitative questions which revolved around student’s perceptions of sttendance, motivations, collaboration, peer interaction, and in-person/remote learning. The insights gleaned are below.

  • In-person/Remote Learning
    Students enjoy the convenience of remote classes but miss interactions of in-person
    “I stay connected through GroupMe, text, and in-person communication”
  • Motivation
    Students utilize friends and peers (even outside of class) to work through problems
    “I ask help from other friends who code”
  • Attendance
    For synchronous resources, students would attend more if considerations were given to scheduling, personalization of material, and reminders.
    “I attended once, and I felt that it wouldn’t help me as much as practicing on my own so I choose not to attend them.”
  • Collaboration
    Mainly positive feeling about collaboration from a social as well as learning standpoint
    “Meeting new people and seeing how other people solve coding problems can give insight into new ways of thinking”
  • Peer Interaction
    There is self-led communication between students within the class
    “I stay connected through GroupMe, text, and in-person communication”

Personas

Who is the target audience?

In order to validate my insights from the previous logic model, I identified three relevant personas to interview. I was able to interview 3 in each category from a wide variety of demographics.

In-Person CS Student
Decided to take the in-person version of the course
Online CS Student
Could not get into the in-person section or could not afford a completely synchronous lecture
Strategic Online CS Student
Strategically took the online version of the class to best suit their needs

Interview Guide

What do I need to learn and validate from the students?

In order to ensure that I would receive relevant, consistent data from my conducted interviews, I wrote an interview guide based on points from my previous logic model.

Findings

Current state of CS 1301's learning community

Based on interview data, in the online section of CS 1301, only 2 out of 3 elements exist: relationships building, and support for different learning styles. As a result, there is no sense of learning community as previously described

Key Insights

What I learned...

The interviews helped confirm my insights from the original logic model and led me to three key insights.

2

Although certain students prioritize flexibility and asynchronicity, talking about problems in real time is a want
“[I attend recitation] to ask the TA’s for help for debugging problems and since it’s the only teaching we get”

3

Past students (in-person section) use CS 1301 as a space to familiarize themselves with course resources and form learning-based friendships which span multiple semesters
“Once I got used to the format they have, taking classes with friends that I met, we would always register for the same classes after”

1

Large majority of students are freshman and are unfamiliar with their learning needs and unaware of what might be helpful
“[As a freshman] I didn’t know what my needs were and didn’t have any wants”

Conclusion

Imperative statement

A good solution should support the development of a learning community, facilitating optional opportunities for student-to-student interaction and personalizing course resources to students’ needs.

Next Steps

Actionable opportunities

Based on research, I was able to identify three actionable opportunities that tied directly to the three discussed elements and would help move CS 1301 toward creating a sense of learning community.

Recitation Schedules

Similar to in-person recitation, If we offer multiple recitation sessions (as opposed to one) we work with the flexibility of students and propose that recitation is a valuable resource.
‍
“More convenient times and frequent announcements [would increase my recitation attendance]”

Collaboration Opportunities

By including facilitated, optional collaboration opportunities within the syllabus, we can cater to those students who enjoy face to face interaction.

‍
“ It can be lonely when you are struggling and simply do not know how to code”

Clarity of Information

Given that this class targets many of the first time CS students, there is a gap in what the supposed benefits of each resource is. By marketing this information, we would allow each student to effectively gauge what might work best for them.
“I wish I was more aware of Piazza and how to find friends in the course”

Final Presentation

Feedback

I was able to present my work to a panel of Georgia Tech faculty. Those present included Dr. David Joyner, Noah Posner, Shawn Harris, and Wayne Li.

This project received immensely positive feedback, and I was encouraged to pursue this work further over the following semesters.

In addition, I was nominated with this project to present at Launchpad, which is Georgia Tech’s College of Design talent showcase.

Acknowledgements

Thank You

I would like to thank Shawn Harris for his time during this independent study. He was able to offer me invaluable advice and guidance as I worked through this project, and these results would not have been possible without his help.

I would also like to thank Dr. David Joyner, my professor under which I was a teaching assistant. It was with his help that I was able to effectively engage with students under the premise of extra-credit activities and optional interviews.

See my work, how I play, or learn more about me
Developed and designed with love and iced mocha lattes
Copyright @ Erin Kingsley 2024
Don't be a stranger
Connect & chat
Kingsley[dot]Erin[at]gmail.com
bit.ly/chat-w-erin
in / Erin_Kingsley