Category Archive: Research

Best Paper Award to Joey Reyes, Undergraduate Intern

We’re pleased to recently learn that Joey Reyes and Monica McGill (advisor) received the best paper award at the 2023 ASEE IL-IN Section Conference held at Southern Illinois University – Edwardsville on April 1, 2023.

Joey, an undergraduate student at Knox College, presented their paper Feasibility of Using the CAPE Framework to Identify Gaps in Equity-focused CS Education Research. This paper describe a pilot test Joey and Monica conducted to determine the feasibility of using the CAPE theoretical framework to identify coverage of equity-focused CSER. The Capacity, Access, Participation and Experience (CAPE) framework developed by Fletcher and Warner examines the capacity to offer CS education, learner access to CS education, learner participation in CS education (enrollment) and experiences learners have when learning CS.

They started with one primary research question: How feasible is it to use the CAPE framework for identify coverage gaps in K-12 CS education research?

Then they created a secondary research question for narrowing down the set of articles examined and testing its feasibility: What are the gaps in research focused on K-12 CS education in which girls are participants in the studies?

They chose to use the Resource Center’s set of 800+ articles and examined studies in which only girls were participants (n=51), then examined each of the 51 articles to determine which key CAPE component(s) each covers. The pilot results showed that CSER among girls covers areas related to Experience (92%) and Capacity (59%), but little to no coverage in the areas of Access (0%) and Participation (2%) of girls. Within experience, coverage in some areas was much higher than others, indicating potential gaps in research.

To answer the primary research question and determine the feasibility of using CAPE for analyzing the entire corpus of 800+ articles, they evaluated feasibility across two key areas, implementation and practicality, and found both to be satisfactory.

Attending SIGCSE Technical Symposium 2023? We’ll be there!

We’ll be actively engaged in the SIGCSE Technical Symposium (TS) in Toronto.

If you’d like to learn more about the work we have recently been engaged in, be sure to stop any of our sessions. Only our workshop requires registration.

Project Announcement: Meta-Synthesis of K-12 CS Education Reports in the US using the CAPE Equity Framework.

CSEdResearch.org received a grant from Google (Google CS-ER) to conduct CS education research. Our project is framed around the notion that, although CS K-12 CS education is growing, there is still a gap in the research, presumably caused by the infancy of CS education research and the barriers around funding research. However, corporations and foundations with a vested interest in improving CS outcomes have made considerable investments in producing reports to illustrate their return on investment. So, while there are reports circulating about K-12 CS education, there has never been a meta-analysis of the information written in the reports. Therefore, an examination of the data and analysis in these reports is the framing of our project.

CAPE, which stands for Capacity, Access, Participation, and Experience, was designed by Fletcher and Warner to examine CS through an equity lens. Therefore, we will use this framework to guide our analysis. Capacity focuses on teachers, funding, and policies that allow for CS education in K-12 settings. Access focuses on equitable access to CS courses. Participation assesses the equitable enrollment in available CS courses. And experience explores variables that affect students’ experiences in a CS learning environment, such as comfort, confidence, and sense of belonging.

Each of the reports chosen for analysis in our study has at least one of these concepts embedded in the report. By examining these reports holistically through the CAPE framework, we will be able to understand the gaps and strengths across the reports. Therefore, the research question we posed in this project is When viewed collectively and objectively using the equity-centric CAPE framework, what will a synthesis of findings from major reports show about equity in K-12 computer science (CS) education?

We hope this meta-analysis will provide a holistic view across the national landscape of K-12 CS education research and build an understanding of how all students, especially underrepresented and historically marginalized students, are accessing, participating in, and experiencing CS education within K-12 settings. We will also provide insight on school, district, and state capacity to offer K-12 CS education equitably. This project will provide a fully published methodology for synthesizing multiple reports, and provide context to other researchers’ findings, whether they align or differ with the synthesized findings.

Written by Angelica Thompson, Senior Education Researcher, CSEdResearch.org

Creating and Evaluating a Research Practice Partnership: Design-Based Implementation Research (DBIR)

Thanks to funding from NSF, CSforALL, CodeCrew, and CSEdResearch.org have created a research-practice partnership (SCRIPTCrewTN) with a design-based implementation research partnership model (Henrick et al., 2017). The work of the RPP centers increasing capacity of school districts in TN to provide equitable access, participation, and experiences among students in K-12 CS pathways. CSEdResearch.org’s role in this project is to conduct formative evaluation research using the Five Dimension of Effectiveness framework to provide feedback to the RPP member organizations so that the RPP can be effective in reaching its goals (Henrick et al., 2017). The five dimensions are:

1. Trust and building relationships,
2. Rigorous research informing action,
3. Supporting the partner practice organization in achieving its goals,
4. Producing knowledge that informs educational improvement broadly, and
5. Capacity building in each partner toward ability to engage in partnership work.

As the internal evaluators, we are focusing on the health and structure of the RPP and evaluating how effectively the RPP:

• Remains centered on equity?
• Identifies and collects pilot year data with an equity lens?
• Trains the facilitators?
• Pilots the workshop with the newly trained facilitators?
• Raises awareness for and recruitment of teachers to take summer CS PD?
• Identifies and collects pilot year data with an equity lens?

Recruitment challenges caused the initial workshop scheduled for February 2022 to be rescheduled for November 2022. We look forward to evaluating and providing feedback on this process.

Written by Angelica Thompson, Senior Education Researcher, CSEdResearch.org

CS Education Research Community: Meet Dr. Melissa Toohey

Dr. Melissa Toohey is the Curriculum Development Director at Seesaw, where she designs rigorous learning experiences that highlight diverse identities to help all students see themselves as computer scientists. Throughout her work she focuses on culturally relevant-sustaining pedagogy that promotes equity and access to computer science (CS) education for all.

Melissa’s passion for equitable education is rooted in her experience growing up as a Taiwanese-American female in a majority-white community. Melissa never saw her identity or culture included in educational experiences, which, as she states, resulted in personal shame and rejection of her culture. Her identity continued to be underrepresented in the CS field, teaching profession, and academia.

As a former Kindergarten and 1st grade teacher, Melissa taught in private, charter, and public school settings and focused on serving students from Title I and historically underrepresented backgrounds. Her greatest accomplishments include establishing and creating the first elementary school CS program in Watts, being awarded a $100,000 grant for 1:1 iPads to increase device accessibility, and helping her students discover their academic and non-academic strengths.

Melissa has led teams inside and outside the classroom to develop CS curriculum and support educators to integrate computer science into daily instruction. Seeing the immense disparities in computer science education and support for educators, Melissa earned her Doctorate from UCLA’s Educational Leadership Program, focusing on equitable Computer Science implementation in elementary school settings. (Go Bruins!)

CSEdResearch.org Connection

Dr. Toohey first found out about CSedResearch.org at CSEdCon 2022, where Drs. McGill and Reinking moderated a session (with fellow panelists Dr. Carol Fletcher and Dr. David Weintrop) focused on sharing key research findings that are applicable to administrators, stakeholders, and curriculum developers. Dr. Toohey stated, “I wish I knew about all of the resources at CSEdResearch.org while I was writing my dissertation! I did a lot of research looking for sources and prior studies, but struggled to find peer-reviewed articles and research around my dissertation topic: Equity in CS Education at the elementary school level. But, had I known about the CSEdResearch.org database, I would have not struggled as much.” 

Although Dr. Toohey only recently found out about our resources, she is excited to introduce our resources to her team at Seesaw. She stated, “since I lead a team to develop computer science curriculum for students nationally and internationally, I know the importance of understanding the research and applying it to my team’s practice. Many other CS platforms and curriculum don’t always take research into account, and I feel strongly that those in research/academia can synergistically work with those practicing and providing educational experiences to students.”

Community listening sessions

Given the recent discussions across asynchronous platforms this past week, many related to feedback from SIGCSE TS reviews, we have set up two listening sessions for community members to talk about their experiences and perspectives.

The listening sessions are an opportunity to share concerns about barriers within the community that prevent researchers from reaching their full potential (and even worse, be driven out of the community) AND ultimately helping the hundreds of thousands of teachers and millions of students that we aim to support.

October 12, 4-5pm CT: To register, visit this link.

October 13, 1-2pm CT: To register, visit this link.

The sessions will not be a time for anyone to offer excuses or rationale for these barriers. Also, you do not need to attend both sessions.

Sessions will be moderated and norms will be set to minimize disrespect and harm. You will be able to share concerns anonymously if you choose. Sessions will be closed captioned. For those with disabilities or conflicts and attending is not possible, please feel free to email me directly with concerns.

We offer this as a community place to recognize the harm as well as the trauma that has been experienced by members in an effort to help move the dialogue into forming actionable steps for improvement at a later time. As an aside, the SIGCSE TS committee has offered to share our aggregated results of these listening sessions on their website through a blog post or another mechanism in conjunction with other plans they have for addressing these issues. Though these listening sessions are independent of the SIGCSE board and SIGCSE TS Committee, we hope they can be used to help inform their future plans. However, we are also very open to hearing more broadly about barriers that go beyond SIGCSE conferences and community.

Connecting K-5 Students to Integrated Computer Science Curriculum

We recently partnered with Code.org to conduct a national study that focuses on how K-5 teachers integrated computer science (CS) into their curriculum. Why? Well, Code.org is working on a new and unique CS curriculum called Computer Science Connections. 

The goal of their curriculum is to teach computing by making critical connections between learning CS and other subjects like math, language arts, science, and social studies. 

Presently, there is minimal research and knowledge available that discusses how and/or why teachers integrate CS into other subject areas. There is also minimal scholarship focused on the barriers teachers and administrators may face when attempting to integrate CS into other K-5 content areas. We believe this will be an important area to watch in the next few years as CS enters into more K-5 classrooms and teachers struggle to balance teaching a new subject without more hours in the day to do so.

Where do we come in? We will be reaching out to states all over the country – 29 in total – to get an overarching view of how and why CS is integrated (or not integrated) into K-5 classrooms. We will also be conducting a systematic literature review (SLR) to better facilitate conversations around promising practices integrating CS into K-5 learning environments. Overall, this information will be used as a launching point that Code.org will be able to use as they continue to expand their mission of teaching all students computer science. 

Are you interested in finding out more about our work? Watch our social media, visit our project page and look for our ongoing updates, or visit Code.org’s Computer Science Connections page and start integrating CS into your curriculum.

Computer Science Teachers’ Problems of Practice: Solve This!

In 2021 we received funding from a ACM SIGCSE Special Projects Grant, with our colleague Dr. Michelle Friend (University of Nebraska – Omaha) for a project we called: Solve this! Problems of practice teachers face in K-12 CS Education. Since then we have been working on gathering, analyzing, and disseminating the findings. Overall, our goal for this project is to provide a platform for researchers to understand authentic problems of practice that teachers face in order to bridge the gap between research and practice. 

What have we accomplished so far?

At the beginning of the project we designed a survey to be sent to teachers around the world. The survey included demographic questions about the teacher and their locale, but most importantly about the problems of practice they experience when planning, teaching, or attempting to plan/teach computer science in their school or classroom. Once the survey underwent internal and external face validity, we disseminated the survey. Our survey reached teachers in Ireland, Canada, and the United States. We opened it in July 2021 and closed it in October 2021, receiving over 700 responses.

After cleaning the data, we were left with 396 responses. We created over 40 codes as we  analyzed the data and several themes emerged. Although we are still in the process of data analysis, some of the initial findings include problems of practice such as a lack of teaching time or schedule availability to teacher CS, poor academic habits, and challenges related to student interactions or partner work. We have been able to share initial results at several conferences and our paper examining our initial set of data has been accepted to Koli Calling 2022. 

What is next? 

Our goal is to have our interactive K-12 CS education teaching problems of practice populated and ready for use by the end of this year. All of the problems of practice entered through this study will be added to our website and will be searchable by demographics of the teachers who submitted them (e.g., country, years teaching CS). 

For researchers, this site will provide you with the problems teachers are facing and can help inform your research agenda. 

Teachers will be able to upvote problems of practice that they experience and will be able to add their own problems. 

Watch our social media platforms for our Problems of Practice page announcement!

Computer Science Teachers’ Personalized Reflection on the CSTA K-12 Teacher Standards

This past summer, CSEdResearch.org had the opportunity to partner with the Computer Science Teachers Association (CSTA) and CREATE (Center for Research on Educational Equity, Assessment, and Teaching Excellence), a research center at the University of California, San Diego, to develop an assessment of teachers’ understanding and use of the CSTA K-12 CS Teacher Standards. As part of this process, we wanted to understand how the Standards can help inform CS teachers’ professional reflection process and their professional development trajectory. With funding from E_CSPD_Week, a U.S. Department of Education EIR grant, CSEdResearch.org joined the partnership to break down the CSTA K-12 Teacher Standards to usable rubric language for personalized reflection and feedback. This summer we piloted a reflection-based assessment for Standards 2-5, with Standard 1 being piloted next summer. After piloting our designed process in two states, Indiana and South Carolina, we learned a lot and continue to improve the process. 

To provide a high level overview of the work that went into the process, our team, along with assistance from CSTA, dissected the CSTA K-12 Teacher Standards 2-5 to create 18 rubric items and scales across three main categories: 1) plan, 2) assessment, and 3) professional growth and development. We then created an entry form to collect the data from a group of teachers in Indiana and South Carolina who participated in the pilot of this work. We are currently undergoing the next phase, scoring and developing a process for external expert readers to provide feedback to the teachers who submit their information as part of this optional process. Our work has resulted in a set of recommendations on how to improve the process so teachers are able to more easily collect and enter their data, which we provided to CSTA and CREATE during a recent discussion. Once completed, this will be tested with a wider group of teachers in summer 2023 and go through a second revision process.

We are also in the process of starting work developing an assessment for Standard 1, CS content knowledge. Working with Dr. Adrienne Decker, we will be creating a brief assessment for AP CS A targeted to high school teachers. We will be piloting this assessment in summer 2023.

Stay tuned for updates on this project. 

Educate Maine: Decreasing Financial Barriers and Increasing Students’ Access to Coding

At CSEdResearch.org we find great value in raising up the voices of our partners who are doing great things in the computer science community. One of those partners is Educate Maine.

This summer Educate Maine’s signature project, Project>Login, hosted 5 Girls Who Code camps all over the state of Maine.

They were able to provide these camps for free, decreasing financial barriers and increasing access for all students.  The girls who participated were able to engage with industry professionals, learn from experienced teachers, and make memories to last a lifetime.

What was our role in this amazing experience? External evaluators. As part of this work, Educate Maine is continuing to reflect and improve their practices through evaluations. The evaluations focused on student and teacher experiences during the week long camps all over the state. Most of the girls who participated in Project>Login’s Girls Who Code camps do not have coding at their school or a Girls Who Code after school program, therefore this summer experience is truly increasing their knowledge of what it means to be a “coder” and, more widely, a “computer scientist”

WEX Industry Partner with Girls Who Code campers

Girls at their Girls Who Code camp

Partner work at one of the Girls Who Code camps