Abstract

The Nationwide Eclipse Ballooning Project (NEBP) is a NASA-supported Science Activation project with a core mission of broadening participation in science, technology, engineering, and mathematics (STEM). The NEBP has engaged 53 teams of students from across the United States in an innovative NASA-mission-like experience with academic scientific ballooning during the 2023 and 2024 solar eclipses. This article reports on evaluation of the NEBP, with a focus on how the project has achieved three primary objectives including: (1) increasing inclusive STEM education opportunities and experience; (2) advancing learner outcomes related to scientific and engineering knowledge, practice, confidence, identity, and interest; and (3) growing and sustaining a network for broadening STEM education. The evaluation evidence suggests substantial success in these areas. Team members indicated that the project had positive impacts on their STEM identities and their interests and sense of competence in pursuing STEM studies and careers. The NEBP also achieved significant goals around broadening participation and creating welcoming team environments, including for students from groups underrepresented in STEM. This article closes with suggestions from partners and team mentors for how a scientific ballooning network might be sustained into the future and a synthesis of lessons learned during the project.  

1. Introduction

The Nationwide Eclipse Ballooning Project (NEBP) is a NASA-supported Science Activation (SciAct) project with a core mission of broadening participation in science, technology, engineering, and mathematics (STEM). To that end, the NEBP has engaged 53 teams of students from a wide range of education institutions in the United States in an innovative NASA-mission-like experience with academic scientific ballooning during the October 2023 and April 2024 solar eclipses. Of the 75 participating institutions, more than 30% are minority serving institutions and 15% are community colleges.1 Most of the participating education institutions are institutions of higher education (IHEs), though five high schools have also participated in the NEBP. This article reports on evaluation of the NEBP, with a focus on how the project has achieved three of its primary objectives including: (1) increasing inclusive STEM education opportunities and experiences; (2) advancing learner outcomes related to scientific and engineering knowledge, practice, confidence, identity, and interest; and (3) growing and sustaining a network for broadening STEM education.

1.1. What is Academic Scientific Ballooning?

Since the early 2000s, scientific ballooning has been a common firsthand STEM learning experience offered in the United States. Academic ballooning uses weather balloons carrying payloads of experiments weighing less than 12 pounds to altitudes of approximately 100,000 feet. At these altitudes, above 99.5% of the atmosphere, payloads experience a space-like environment. Equipment types and purposes of scientific ballooning can vary, though they generally involve data collection. NEBP teams were one of two types: (1) science teams launching atmospheric science radiosondes or (2) engineering teams launching student-built platforms with varying payloads and instrument packages. Radiosondes are small, standardized, off-the-shelf payloads of less than 190 grams that are used to measure parameters throughout the stratosphere. Typical radiosonde-based atmospheric science investigations examine phenomena such as gravity waves, planetary boundary layer changes, and weather patterns. Engineering balloon platforms in the NEBP were capable of lifting up to 12 pounds of student-built payloads into the stratosphere. Typical engineering platform experiments and investigations focus on atmospheric measurements, imaging, cosmic radiation measurements, and space technology proofs of concept.

1.2. Why Engage Students in Scientific Ballooning During the Eclipses?

The NEBP was designed to leverage natural excitement around rare solar eclipses as a means for engaging diverse participants in extended, learner-centered, and team-based scientific missions that actively develop their STEM knowledge, skills, interests, and identities. Through joining NEBP teams, students have had the opportunity to work with subject matter experts to develop deep STEM knowledge; to design, test, and build scientific ballooning systems; to fly eclipse balloon campaigns collecting high impact data; to analyze those data; and to present and publish their results. Data collected by teams in the two NEBP tracks (atmospheric science radiosonde and engineering platform) are informing the investigation of important scientific and engineering questions including, for example:

1. Can eclipse-induced atmospheric gravity waves be definitively detected in data across all sites?

2. What is the magnitude of the temperature drop during an eclipse at the surface and in the planetary boundary layer, the troposphere, and the stratosphere?

3. To what extent is the kinematic response of the surface wind field within the path of totality time-lagged to the thermal response?

4. Can current high-resolution weather forecasting models simulate the observed responses and, consequently, improve weather modeling physics and forecasting?

The NEBP design operates on the idea that engaging in the project’s team-based mission activities can provide an on-ramp to continuing STEM studies and careers for those who may be interested in but new to science and/or engineering. Fewer than 40% of students who enter college intend to complete a STEM degree, and even fewer actually complete the degree. To meet anticipated workforce demands, the numbers of people studying STEM and entering the STEM workforce will need to increase (Harrell & Capco, 2021; NCSES, 2023; Olson & Riordan, 2012). Team-based and experiential learning opportunities have been shown to have positive impacts on recruiting and retaining students in STEM fields (Harrell & Capco, 2021; Jin et al., 2019; Tsui, 2007). Connecting scientific ballooning campaigns in academic settings with excitement around the 2023 and 2024 eclipses provided a unique opportunity to draw new students and new institutions into a growing and evolving community of STEM. The team- and mission-based activities of the NEBP deemphasize learning about science and engineering in a classroom while foregrounding active participation in professional activities of science and engineering to expand human understanding.

The NEBP design intends to provide students with an exciting STEM workforce development opportunity and to broaden participation of underrepresented learners, consequently helping to build an increasingly diverse community of STEM professionals. The NEBP has worked to broaden participation not only by encouraging a high level of diversity for teams at four-year research IHEs, but also with an emphasis on building teams from minority serving institutions (MSIs) and community colleges (CCs).

1.3. How Was The NEBP Designed to Work?

1.3.1. Formation of the NEBP Team Network

In an effort to accomplish its objectives, the NEBP has continued to connect with and help grow the nationwide network of organizations offering academic scientific ballooning. The 53 NEBP teams participate in one of nine regional pods where they have benefitted from access to a pod leader institution that offers technical support and facilitates communication with each of its pod’s teams.

The NEBP leadership team and pod leads were established by drawing on team leads from earlier 2017, 2019, and 2020 total solar eclipse ballooning projects as well as engaging experts in stratospheric ballooning and atmospheric science. After a planning period in 2021-2022, the NEBP solicited proposals from prospective teams from across the country in fall 2022. NEBP planning, partnership building, and promotion also leveraged the institutional network of the National Space Grant College and Fellowship Project. The director of the NEBP leads a state’s Space Grant Consortium and many of the teams include mentors affiliated with their state’s Space Grant Consortium.2 Additionally, many of the NEBP teams have received supplemental funding resources from their state’s Space Grant Consortium.

Most of the team applications were submitted by STEM educators (e.g., faculty members) at educational institutions. The NEBP leadership team and pod leaders provided guidelines and informational resources (including a webinar) to potential teams during the solicitation phase, helping them to address specific topics within their proposals that went beyond just technical details. Team proposals needed to include a mentoring plan; resource statements; a diversity, equity, inclusion, and accessibility plan; a sustainability plan; and team assurances.3 Proposals required support from the institutions’ Authorized Organization Representative to ensure teams were financially supported and able to participate.

MSIs were prioritized in the institutional recruitment phase. Also, recruitment information specified that MSIs and CCs would receive additional travel funds for participation. The NEBP sought to include as many teams as possible and encouraged multiple institutions in close proximity to combine into single teams with greater reach. Every institution that submitted a proposal was selected to join the project. Once selected for participation, team mentors recruited student team members through multiple channels such as course announcements, advising, and state Space Grant Consortium communications.

1.3.2. Overview of NEBP Team Activities

The NEBP teams of students have pursued research in atmospheric science or engineering working alongside mentors, NASA subject matter experts (SMEs), and other STEM professionals. In total, more than 750 students and 150 team mentors have collaborated in this 18-months long, mission-based project that has the potential to produce over a dozen peer-reviewed journal article submissions, a sustainable nationwide network, and life-changing experiences for participants.

As mentioned, atmospheric science teams have flown radiosondes while engineering teams have flown balloons carrying payloads engineered by students. Each NEBP atmospheric science team flew up to 30 radiosondes per solar eclipse, launching a balloon every hour beginning 24 hours prior to a solar eclipse and continuing until six hours after (Figure 1). NEBP engineering teams launched just one or a few balloons during each eclipse, generated real-time video (streamed publicly), made high-resolution GPS measurements to complement the radiosonde data, and conducted other applicable individually designed experiments involving, for example, 360° cameras and radiation sensors (Figure 2). Both atmospheric science and engineering teams collected data to inform the scientific study of how the atmosphere reacts to the cold, dark shadow of a solar eclipse.

In addition to core ballooning mission activities, the NEBP structure has provided additional resources and opportunities to teams. Examples include: (1) regional training workshops organized by pod leads to prepare mentors for team leadership, (2) NEBP online course materials that teams could use to implement credit or non-credit bearing courses or in an informal learning context, (3) guidelines and responsibilities for teams to carry out eclipse and ballooning related educational outreach, (4) instructional advice to help team participants create a career portfolio, (5) a STEM professional speaker series highlighting career possibilities and facilitating professional connections, and (6) NEBP community-building activities such as all team webinars and an online discussion forum.

The NEBP’s national network model has the potential to serve as a framework for replication (Saad et al., 2024). Distinct from typical short-term STEM projects, the NEBP provides a model for long-term, mentor-intensive, enduring efforts that strive to engage students in unparalleled real-world, mission-based STEM experiences. The scientific research the NEBP is conducting can only be done with a large group of people (i.e., more than 500 individuals) to cover many sites with high temporal resolution. Through project implementation, the NEBP strategically brings together practicing scientists and (primarily undergraduate) students to collaborate on innovative research. The students have carried out field campaigns and data analysis via NEBP-provided courses and in-depth hands-on practice.

2. Evaluation Methods

2.1. Evaluation Design and Questions

NEBP evaluation adopted a participatory approach with regular coordination and communication between the evaluation team and the project leadership team (O'Sullivan, 2004). During most of the project timeline, efforts have emphasized iterative formative evaluation to inform refinements during project enactment. Evaluation has also included a summative component aimed at documenting and reporting project effectiveness in achieving its objectives (Frechtling et al., 2010).

In this article, we draw mostly on evaluation data collected from student team participants, team mentors, and other NEBP partners after the April 2024 eclipse. We use these data to provide a summative report on: (1) the ways and extent to which various participating groups perceived that the project achieved key objectives, and (2) insights and suggestions that were offered for sustaining and building on the NEBP’s accomplishments into the future.

Evaluation questions addressed include:

1. In what ways and to what extent did participants, team mentors, and core project partners perceive that the NEBP achieved its objectives around:

   a. increasing inclusive STEM education opportunity and experience;

   b. advancing learners’ scientific and engineering knowledge, practice, confidence, identity, and interest; and

   c. growing and sustaining a network for broadening STEM education.

2. What interests, insights, and suggestions were offered for sustaining and building on the NEBP’s accomplishments into the future?

2.2. Participants and Data Collection

The evaluation process has incorporated surveys administered at three key time points: (1) project teams’ start, (2) after the 2023 annular solar eclipse, and (3) after the 2024 total solar eclipse. Customized survey forms have been implemented via the Qualtrics survey platform for different groups including core partners, team mentors, and student team members. These surveys have sought to capture diverse perspectives on how the project is working.

The design of the surveys was guided by a need to evaluate the project’s achievement of proposed objectives. Thus, for student respondents, the surveys have asked about personal and project-related perceptions and experiences related to engagement, teamwork, mentorship, learning, skills, diversity equity inclusion and accessibility (DEIA), STEM identities, and STEM studies and career intentions. Surveys for partners and team mentors focus on domains such as collaboration, capacity building, needed support for effective project implementation, and perceived project impacts. All of the surveys included both closed-ended (e.g., Likert scale type) and open-ended questions.

Reports from the surveys have been used both for formative purposes (i.e., to inform responsive decision-making during project implementation) and summative purposes (i.e., to report to NASA the ways and extent to which the project is meeting its stated objectives and outcomes).

While response rates for surveys of core partners and team mentors have been quite good (80% to 90%), survey response rates among student team members have been consistently lower (35% to 50%).4 A few explanations likely underly the lower response rates for student team members. Prominently, the NEBP is a largely decentralized network project in which communications between the project leaders and the student team members have been mediated by team mentors. The NEBP leadership and evaluators did not have a list and contact information for all the student team members. Team membership was also fluid given the length of the project—with mentors reporting that many team members left (e.g., because they graduated) or joined throughout the 18-plus months of active student participation in NEBP activities. Some team mentors also conveyed that student participants were often extremely busy, overloaded, and hard to get in touch with, particularly in the month following each eclipse (which was also the collection period for two of the three survey implementations). For the final student survey, we encouraged team mentors to provide time for students to complete the survey during a post-eclipse team debrief meeting (as we have found that response rates increase when evaluation activities are implemented during time allocated in in-person meeting agendas). However, we do not know how many teams followed this suggestion.

While we initially planned to implement evaluation with student team members through a series of the surveys that would be matched by student across time points, the low response rate for students made it clear that this approach for collecting evaluation data would not work well.

A related challenge was that many students preferred not to provide identifying information in their survey responses, which made matching surveys by student over the course of the project difficult. Similarly, many students preferred not to provide demographic data (e.g., race, ethnicity, gender) about themselves. Survey language was clear that students could choose which questions they wanted to answer, and that data would be shared with project leadership in a de-identified, summarized report format. An explanation was also provided in the survey for why demographic questions were asked and how the demographic results from the survey would be used (including to examine whether the objective of broadening participation among groups underrepresented in STEM was achieved).

In response to these issues (and related ones such as infeasibility of implementing a random assignment, experimental design evaluation study), the post-eclipse surveys were designed with wording to query participants about their perceptions of the project’s impacts (Friedman et al., 2008; Robson, 2024). This design allows us to make claims about respondents’ reported perspectives concerning how well the project worked at accomplishing its intended objectives. In addition, the evaluation findings were supported by complementary data sources such as leadership-initiated reporting by team mentors about their project activities and accomplishments.

2.3. Analysis

For closed-ended survey items from all project groups, simple analyses were undertaken to generate descriptive statistics and frequencies. Responses to open-ended questions were reviewed to identify prominent themes and example quotes were selected to provide qualitative illustrations of how project participants viewed their experiences. For questions that were implemented as retrospective pre-post format, basic inferential statistics (i.e., paired samples t-tests) were applied to examine differences between retrospective pre and post responses.

3. Findings

3.1. Increasing Inclusive STEM Education Opportunity And Experience

In this section, we focus on respondents’ reports about the opportunities and experiences provided by the NEBP. In other words, what activities did responding students report engaging in during their NEBP participation and what views did they express about those activities?

3.1.1. Participation in NEBP Activities

Figure 3 shows that student team member survey respondents reported high participation rates in some project activities and moderate and lower rates in others.5 Most respondents reported participating in their teams’ eclipse activities, which usually required travel to the path of totality. More than half of respondents reported taking a project-related course, either for or not for credit (54%). Moderate percentages of students reported participating in an outreach event (44%). Fewer students attended an online “Between the Eclipse Research Conference” organized by the project (13%) or worked on a project-related scientific manuscript (8%). These responses suggest that there was variation in how students participated in the project. Comments from team members and mentors (both within surveys and in other communications) suggest that some participants encountered challenges such as limited time due to school and personal commitments (e.g., childcare), but this information was not systematically collected and other reasons such as lack of interest in particular activities are possible explanations as well.

3.1.2. Opportunities Afforded by NEBP Participation

Team members shared their views about the opportunities that the project offered (Figure 4). 79% of respondents reported that, to a great or very great extent, participation facilitated their joining a network of people with varying levels of STEM experience. 58% reported that, to a great or very great extent, the project provided an opportunity to participate in activities that benefit people and communities.

Team members were also asked an open-ended question about what they found most memorable or valuable about their project experience. Prominent themes among responses included viewing the total eclipse; being part of a team of colleagues in STEM; engaging in real-world research; and developing STEM knowledge and skills that will support them in attaining their academic, career, and personal aspirations. Students’ responses often interwove multiple themes together. For example, one student wrote, “Working and interacting with people that have much more experience than I do and the feeling of welcome and value that they transmit to me. Being a part of something that is nationwide and affiliated with groups like NASA have expanded my interest and broadened my exposure beyond anything I have ever imagined.”

3.1.3. Inclusivity of NEBP Team Environments

As a project seeking to broaden participation and invite students from underrepresented groups into the community of STEM, it was important to examine not just who participated in the project, but also the extent to which students positively viewed their experiences. Most respondents agreed or strongly agreed (≥89%) that their team environment was collaborative, friendly, and welcoming to diverse members (Figure 5). Students’ positive views of their team environments were also evident in their comments about what was valuable to them. One respondent wrote that they valued, “[t]he team building aspect. For our team, we all learned distinct roles and responsibilities, we had to work as a unit, over time and learning each other's communication styles we melded as a team. That was a valuable experience to learn how to make a team out of a diverse bunch of people.”

Mentor surveys from earlier in the project as well as communications with mentors more generally suggested that these positive outcomes were likely due to a combination of many mentors who were already well-versed in creating a welcoming STEM community environment as well as emphasis on this issue and guidelines for effective practice provided by the project leadership.6 The project website included a DEIA page with information and links to further resources like the Breakthrough Inclusive Action Toolkit (Science Friday & HHMI, 2020). In addition, the online project course materials emphasized not just technical skills but also exemplary practices for DEIA in STEM. The very low percentages of respondents who had negative views of their team environments (<5% for “collaborative”, <3% for “friendly and welcoming”, and <2% for “non-racist, non-sexist, and non-homophobic”) is encouraging, but highlights that continuing efforts are important to try to reduce negative experiences for all young adults entering STEM community environments.

3.2. Advancing Learners’ Scientific And Engineering Knowledge, Practice, Competence, Confidence, Identity, and Interest

Next, we turn to respondents’ views of how the project supported key objectives for team members as learners. Most of the evidence comes from the team participant survey, though we also include team mentors’ views of the project’s impact on these STEM development outcomes.

3.2.1. STEM and STEM Career-Related Knowledge, Practice, and Competence

Figure 6 and Figure 7 show how respondents reported that participating in the project helped them develop their STEM-related knowledge, practice, and competence. Most respondents reported that, to a great or very great extent, participation supported them in deepening their knowledge in STEM topics (68%) and their understanding of diverse perspectives in STEM (60%). Not quite half the respondents reported, to a great or very great extent, that participation supported them in developing capacity to use STEM to address societal issues (46%). Responding participants robustly agreed that they increased their competence in areas such as STEM teamwork, STEM research, tackling an engineering problem, establishing group norms, and leading a group of peers in STEM (Figure 7). Reflections on their growing sense of personal competence could also be seen in many of the students’ written comments, such as one student who noted valuing, “The community of people surrounding me. It made me feel like I deserved to be there, and that I will do well in a collaborative environment in the future with my peers.” Another respondent noted that the project, “…gave me more confidence in my academic journey.”

3.2.2. Job Attainment Confidence

The survey included a retrospective pre-post query concerning team members’ confidence that they could get the job they desire after graduating. 44% of respondents expressed that they were confident or very confident just before they joined the project while 78% expressed that they were confident or very confident at the time they took the final project survey (after the April 2024 eclipse) (Figure 8). Using the survey item scale, which spanned from 1=very unconfident to 5=very confident, we ran a two-tailed, paired samples t-test to compare mean retrospective pre (3.32, SD=1.05) with mean retrospective post (4.06, SD=0.82) [t(311)=12.76, p<0.001].

Promisingly, less than 3% of respondents reported they were currently unconfident or very unconfident (compared with 21% reporting that for the retrospective pre question). With respect to career confidence, some students expressed that the project had a transformative effect on their views. One student, for example, noted the impact of, “[t]he diversity of topics and variety of exposure. I am amazed at the opportunities that have come my way because of my STEM / NEBP participation. Knowing what is available to [do] and how it can be achieved leads to brighter futures for myself and my family.”

3.2.3. STEM Identity

The retrospective pre-post item about STEM professional identity overlap (McDonald et al., 2019) was one that the SciAct program requested as a way to establish a shared measure of STEM identity across projects in the program.

The questions ask:

Retrospective pre: Think back to the time just before you joined the project and select the picture that best describes the overlap of the image you had of yourself and your image of what a STEM professional was.

Post: Select the picture that best describes the overlap of the image you currently have of yourself and your image of what a STEM professional is.

The questions are accompanied by a set of seven choices showing two circles with varying amounts of overlap: almost no overlap=1, very small overlap=2, small overlap=3, moderate overlap=4, large overlap=5, very large overlap=6, almost complete overlap=7 (Figure 9). For the retrospective pre, the mean response was 4.01 (SD=1.77). For the current (post), the mean response was 5.13 (SD 1.37). The two-tailed paired t-test result was [t(312)=16.72, p<0.001]. On average, respondents reported a shift from a moderate overlap in identity to a large overlap in identity over the course of their participation in the NEBP. Stated another way, on the retrospective pre, 37% of respondents reported at least a large overlap whereas on the post, 69% of respondents reported at least a large overlap (Figure 10).

3.2.4. Interest in STEM Studies and Careers

More than half the respondents reported that project participation led to an increase in their interest in STEM studies and a STEM career (Figure 11). Further, almost all the respondents who did not report an increase in interest reported that their interest level stayed the same. Less than 1% of respondents indicated that their interest decreased. Given the nature of the project and the commitment required of participants, it is likely that those who indicated that their interest level remained the same were already thinking about STEM studies and careers when they joined a project team.

Students described the importance of having a mission-based, real-world project, sharing reflections such as, “[w]orking on the total eclipse project allowed me to see the tangible impact of theoretical concepts, igniting my passion for pursuing a career in STEM.” Challenges like the uncertainty that many students feel about pursuing a STEM career, as well as the boost to stay in STEM that can come from a positive experience, were also evident in comments. For example, one student wrote, “I learned a lot about research projects, and it has caused me to reconsider switching out of the STEM field.”

3.2.5. Team Mentors’ Perspectives About How the Project Impacted Participating Students

Surveys asked team mentors (including partners who were also team mentors) to report the extent to which they felt their institution’s participation supported a variety of outcomes for team members (Figure 12). More than half of team mentors agreed with all outcome statements either to a great or a very great extent. Mentors strongly endorsed that participation supported students in increasing STEM knowledge and skills and developing identities as STEM professionals. While mentors almost all expressed that participation supported, at least to a small extent, increasing understanding of STEM careers and of diverse perspectives of science, these outcomes were less robustly endorsed.

3.3. Growing and Sustaining a Network for Broadening STEM Education

Given that there are multiple facets to this objective (i.e., growing a network and broadening participation) we provide a few forms of evidence of project effectiveness in this area including: (1) perspectives about institutional capacity building and (2) participation by students from groups underrepresented in STEM.

3.3.1. Building Institutional Capacity

The team mentor and project partner surveys included questions about the extent to which these groups reported that participation increased their institution’s capacity in key areas (Figure 13). Most mentors and partners agreed to a great or very great extent (>71%) that participation increased their institution’s capacity in all areas about which they were queried. Particularly strongly endorsed were the capacity to carry on scientific ballooning activities in the future, to undertake multi-institution ballooning collaborations, and to offer inclusive and equitable STEM education experiences. The mentors and partners also reported increased capacity in supporting students on STEM career pathways and engaging students in research involving remote sensing.

3.3.2. Participation of Those From Groups Underrepresented in STEM

The NEBP proposal identified a goal of recruiting at least 50% of its participants from groups that are underrepresented in STEM. Underrepresentation in STEM was defined in the project as self-identification in one or more of the following groups:

• Hispanic or Latino/a

• A race/ethnicity other than or in addition to White or Asian

• Gender identification other than male (responses included female, genderqueer, non-binary)

• Member of first generation in family to pursue a college degree

• Experience of family financial hardship while growing up “always” or “most of the time”

Of 269 participants who provided enough demographic information to examine the criteria, 172 (64%) indicated identification with one or more of the underrepresented group criteria. If we assume that all of the respondents who did not provide any or sufficient demographic data did not identify as belonging to any of these groups, the percent of respondents who identified as belonging to an underrepresented group still exceeds 50% (172 out of 321 or 54%).

The NEBP’s success with respect to engaging participants from groups underrepresented in STEM may relate to the project’s explicit efforts to recruit MSIs to host teams and to encourage all team mentors to recruit student participants from underrepresented groups.

3.4. Insights For Sustaining and Building on The NEBP’s Accomplishments

The post total eclipse surveys all included some questions asking for reflections and advice for continuing the academic ballooning network. Here we summarize some key feedback and interests expressed by project partners and team mentors.

The partner and mentor surveys queried respondents about their interest in continuing involvement. The project leadership was gratified to learn that most of the partners and team mentors expressed interest in continuing involvement in an academic scientific ballooning network. 92 of 105 partners and mentors (88%) who responded to a question about future involvement indicated that they would either “probably” or “definitely” be interested in continuing.

The team mentor and partner surveys also included an open-ended question asking, “If the NEBP were to evolve into a network that continues to support teams across the country in scientific ballooning activities, what recommendations do you have for how that network could be successful and provide a great experience for future students?” Table 1 summarizes some of the prominent themes among suggestions offered by team mentors and partners. These suggestions will be useful for the project leadership as they consider crafting a proposal for a next phase of nationwide, networked academic scientific ballooning.

Recommendations for an Ongoing Academic Ballooning Network

4. Discussion

The NEBP evaluation has provided formative evidence that has informed refinements to the project during implementation, summative evidence that has supported the leadership in reporting on success in achieving project objectives, and insights for designing a successful next phase of a nationwide, academic scientific ballooning network. Here we share some of the lessons learned from project implementation and evaluation—in particular lessons that will be carried forward to future efforts. We also discuss several limitations of the project and evaluation.

4.1. Lessons Learned

To be successful, a complex and diverse network like the one created by the NEBP needs to manage a careful balance between creating systems and infrastructure (i.e., modus operandi) while also allowing for flexibility. With this balance in mind, important lessons that emerged include the following:

1. Institutions/teams appreciate the systems and resources that the NEBP provided, which allowed even new teams to jump in to carrying out a complex and ambitious mission with their students. At the same time, different institutions and teams need varying levels of support depending on their available resources and levels of experience/expertise. Support needs vary, but often include funding, research, technical and equipment, logistics, and curricular materials.

2. The need for balance between set systems and flexibility is also evident in the area of communications. There was a strongly expressed need to receive clear communications from the project leadership. At the same time, there was also a desire to facilitate more inter-team and inter-student communications, so both top down and horizontal communications channels are important.

3. Academic ballooning campaigns and activities need to be sensitive to students’ busy schedules and many competing demands for time. This is especially relevant for students with family responsibilities and/or fewer monetary resources who may be balancing things like jobs and childcare. Flexibility may be necessary to acknowledge that different students and teams may not complete every activity that is offered, and that that is okay.

4. The NEBP experience suggests that a combination of intentionality, guidance, and support for teams and their mentors; acknowledgement of existing expertise; and flexibility can contribute to helping teams create a welcoming STEM community environment as perceived by student participants. While they appreciate relevant resources, many mentors already have deep experience and capacity in this area, and it is important to acknowledge this expertise rather than trying to impose particular protocols around DEIA. Flexibility with regard to DEIA practices is also increasingly necessary given the politically charged climate around DEIA efforts in higher education in the United States (Feder, 2024).

5. We found that the NEBP’s distributed network (balancing structure and differentiation) holds promise for offering unique opportunities for collaboration, creativity, and cross-team mentoring. The network provides resources and latitude so team mentors, with help from pod leaders, can figure out how to tailor efforts toward various objectives (e.g., making a place for students in a professional network, providing real world STEM experiences, building students’ confidence in pursuing STEM studies and career) for their particular students.

6. Finally, while the two rare solar eclipses were an excellent catalyst to spark interest in academic ballooning, eclipses are rare events. An ongoing nationwide academic ballooning network will likely need to identify a new focus or set of foci given there will not be another total solar eclipse visible in the contiguous United States until 2044!

4.2. Some Limitations of the Project and Evaluation

Here we highlight a few limitations of the project and evaluation that the NEBP leadership has been reflecting on. One possible project limitation has been the variation in engagement across different activities. While most team member respondents indicated that they participated in their teams’ eclipse activities, fewer students engaged in other project activities like working on a career portfolio, presenting their project work at a conference or meeting, or contributing to a project-related scientific article. We would like to explore these differences further to address some continuing questions. For example, it would be helpful to have a better sense of to what extent varied participation among students may reflect differential resources (e.g., available time and money) versus differential interest or other causes. Further evaluation efforts could help leadership decide whether to generally be okay with varying levels of participation (i.e., to the extent they reflect varying levels of interest in “optional” activities), or if leadership should put even greater emphasis on directing resources to teams and students with fewer resources available to them.

The evaluation data collected as part of the NEBP provides evidence that the project worked largely as it was designed and was able to achieve its stated objectives to a significant degree. One limitation of the evaluation is that we were not able to achieve a response rate above 50% among the student team members. Though team member response rates were lower than desired, the demographic questions did provide evidence that the respondents represented a diversity of students in areas such as race, gender, and socioeconomic status.

The evaluation was also limited by the infeasibility of utilizing experimental methods such as randomized control groups and connecting data across multiple time points to support causal claims about project impacts. The evaluation team responded to this limitation by designing data collection instruments that queried participants’ perspectives about the project and how it worked. This approach allows us to make reasonable claims about how various groups (team members, mentors, partners) perceived the project and its impacts. Based on both descriptive statistics indicating positive views of the project and open-ended comments providing insights into personal highlights, the evidence suggests that most survey respondents had favorable views concerning both how the project worked and the extent to which the project achieved its objectives.

5. Conclusion

The NEBP’s multifaceted approach to scientific ballooning-based education and research, along with its commitment to DEIA and team-based learning environments, positions it as an innovative example for scalable, long-term, STEM programs. The evaluation evidence suggests positive impacts on primary objectives including: (1) increasing inclusive STEM education opportunities and experience; (2) advancing learner outcomes related to scientific and engineering knowledge, practice, confidence, identity, and interest; and (3) growing and sustaining a network for broadening STEM education.

The NEBP is currently undertaking and planning for continuing efforts in several areas. Scientific data analysis and research dissemination activities utilizing data collected during the 2023 and 2024 eclipses, as well as several new areas of scientific ballooning-based inquiry, will continue with participation by students, mentors, and science matter expert partners, at least through 2025. The project leadership and partners, with input from across the network, are also envisioning what a next phase for the network might look like in preparation to submit a proposal to an upcoming notice of funding opportunity. With respect to evaluation, and to provide insights for future efforts, the evaluators are also planning to dig deeper into the data to explore whether groups of students (e.g., those in MSIs and CCs versus non-MSI or CC institutions, or those who participated in activities to a lesser or greater extent) differed in how they perceived the project and its impacts.

Evaluation evidence suggests that the project design has helped equip the NEBP with a resilient and adaptable foundation that can offer other STEM programs a model for scalability toward broadening STEM participation, building and sustaining nationwide STEM learning networks, and bridging the gap between STEM education and firsthand scientific research. Beyond sharing the achievements of this project, we hope that this evaluation report may offer valuable insights for other projects and organizations operating in similar domains and with similar goals for STEM participation and capacity building.

Acknowledgements

This research is supported in part by a grant from the National Aeronautics and Space Administration for the Nationwide Eclipse Ballooning Project (Award number 80NSSC22M0003). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Aeronautics and Space Administration.