Skip to main content
SearchLoginLogin or Signup

Identifying and Analyzing Astronomers' Preferences for the Next Decade of NASA

In this research, we survey preferences in the astronomy and astrophysics community as they relate to the next decade of NASA. We elaborate on community preferences by presenting results on mission concepts, mission capabilities, and primary science.

Published onApr 14, 2021
Identifying and Analyzing Astronomers' Preferences for the Next Decade of NASA
·

Abstract

The aim of this research is to survey preferences in the astronomy and astrophysics community as they relate to the next decade of NASA. Preferences are key for understanding how decisions are made by individuals that could aid in projecting community preferences and likely outcomes. The study was performed using a survey methodology and evaluated preferences regarding the decadal survey by prioritizing space-based astrophysics missions and research activities. Thematic analysis was implemented to determine themes within participants' open survey responses. Our results show strong community preferences for observational astronomy and science versatility, capability, and scientific return for the upcoming decadal period. Moreover, our findings also show consensus among subgroups in the community. We elaborate on community preferences by presenting results on mission concepts, mission capabilities, and primary science that NASA should strategically invest in.

1. Introduction

The NASA Astrophysics 2020 Decadal Survey is a summary of consensus recommendations from individuals in the astronomy and astrophysics community to define the next decade of prioritized missions and activities. The study, which is conducted by the United States National Academies of Sciences, Engineering, and Medicine, generates consensus recommendations to develop a comprehensive strategy and vision. The survey addresses past and present research as well as changes in civil policy, new programs, the effects of cost and schedule, topics of interest, and how the community may respond to the results [1]. Each of these aspects are considered by an ad hoc survey committee that receives input from study panels to cover the breadth of decadal studies and develop an overarching strategy to advance the frontiers of astronomy and astrophysics for the 2022-2032 decadal period.

From a psychology and systems engineering perspective, the decadal survey process is interesting because of the large and diverse community being sampled, the potentially diverse preferences held by individuals, and the committee’s interactions that result in a recommendation. When preparing input to be reviewed in such a process, it is important to recognize and understand individual as well as community factors. By understanding these dynamics, it is possible to project the likely outcome.

From a practical perspective, the ability to gauge a likely outcome can steer the presentation of projects to best suit what the community values. Every project is multi-faceted and complex; therefore, knowing the community’s interests provides the ability to showcase how a mission concept meets the community’s needs and values. Knowing that a community values durability, for instance, would allow a mission concept to illustrate how the plans have taken durability into account to better inform the community. Thus, the aim of this research is to contribute knowledge regarding the nature of preferences held by individuals in the community and the amount of consensus between subgroups in the astronomy and astrophysics community.

In addressing these issues, we conducted a study to collect community member’s preferences ahead of the 2020 decadal survey that weighted the suggestion of individual committee members and panels of the community to implement a strategy and vision for research and funding. This study contributes to the understanding of community preferences concerning the upcoming decadal study; importantly, it elaborates on preferences with insight into community reasoning, perceptions, and interests in NASA science missions.

1.2 Preferences Formation

Preference formation is a key principle for understanding how decisions by an individual are made that could aid in the projection of community preferences and likely outcomes. Research defines preference as the buried inclination an individual feels that determines whether something is desirable or not [2]. Underlying preferences are stated by utility theory to be complete and stable across contexts, such as the comparison of different mission concepts; however, other research suggests that preferences are not only moderately unstable, but are addressed and adjusted during individual and collective decision making processes as well [2]. The context of the decadal survey goals affects what preferences are formed, such as when the goal is to develop a well-balanced strategy that considers large, medium, and small activities for both ground and space or to utilize decision rules for a comprehensive research strategy [1]. Thus, a decision made is based on preferences formed to meet those goals rather than weighing in other options. Moreover, the complexity of identifying, recommending, and ranking the highest priority research activities determines how much calculation is needed to form a preference; the more information that needs to be considered and evaluated, the more calculation is required to form a complete preference [2]. Given that a decadal survey is a culmination of responses generated under relatively time-sensitive motivation within a given context, not all decisions have excess time available for preferences to be calculated. The intensity of the time constraints relates to the likelihood that individuals may rely on heuristics to form their preferences. Although individuals make use of heuristics to access previously formed preferences, the more familiar the individual is with the domain of the research activities, the greater the chance they have already created preferences to fall back on and help alleviate heuristic thinking [2]. Regardless of the degree to which heuristics are used, the expressed preferences ultimately lead to the decisions that an individual makes.

1.2 Decision Formation Regarding the Next Decade of NASA

The purpose of reviewing literature on individual preferences is to understand how individuals form a decision regarding the decadal survey. Classical theory provides the simple conclusion that since decisions are hedonistic in nature, they are made to maximize the utility to the individual and minimize consequences [3]. Each decision the individual makes will need access to the subjective probabilities of what they think their decision could result in. There is a possibility that a situation requires a different set of preferences that would need to be met, but not one decision will satisfy all. In these cases, each preference is weighed to determine which decision provides the balance of maximum utility and minimum consequence [3]. As with the formation of preferences depending on heuristics in decision making, the preference that is most cognitively retrievable is usually the one that is more desirable and thus most influential in the decision-making process [4]. Context affects decisions in the same way it affects preferences. Even something as simple as reaching a conclusion in the same physical location that the work was done could trigger contextual cues that make self-contained preferences more influential and more accurate to the needs of the specific group [4]. To further understand preference formation, community preference among institutions is explored.

1.3 Preferences amongst the Community and Institutions

In contemplation of what the astronomy and astrophysics community values regarding the upcoming decadal period, we first need to consider what is meant by ‘community’. According to Raco [5], the term ‘community’ holds different meanings across discipline, time, and space. For the purpose of this study ‘community’ is considered as a network of ‘like-minded’ individuals, regardless of geographical, political, cultural, or other boundaries [6]. Preferences of a community in this study can be understood as an aggregated picture of shared preferences held by individual members of the community regarding the decadal survey. As such, only preferences that are observable, those expressed verbally or through writing, are indicative of consensus among community members. During collaborative decision making, however, preferences of individual members and the group itself may reach their polarization from the reinforcement of these preferences [7]. As the decision is made by the group, individual members become more confident in their decision and accordingly adjust their preferences in a cyclical affirmation of their beliefs. Although the same principles apply with demonstration and polarization bearing weight within communities, all members of a community cannot discuss information and come to a community decision. Instead, communities such as the astronomy and astrophysics community are made up of numerous groups with their own values and decisions that may reflect the most common and strongest values.

In behavioral research, universities and institutions may not always be considered representative of the general population because of inexperience, lack of knowledge, or biases in convenience sampling. Community preference data is unique in that data from every individual is important for the complete representation of the community. The purpose of the decadal survey is to generate consensus preference data, provide an overview of the current state of astronomy and astrophysics science, and develop a vision for the future by prioritizing research activities. According to Chang and Huang [6], institutions are the main driver for scientific research. Therefore, our study is beneficial to the astronomical and astrophysics community by providing a preference list by taking into consideration industry and academic institutions as a projection of community interest for the coming decade.

1.4 Research Objectives and Contribution

This study aims to better understand preferences of the astronomy and astrophysics community as they relate to the NASA Astrophysics 2020 Decadal Survey. By understanding preferences and making assumptions on beliefs and available alternatives, it can be determined what decision an individual ought to make and steer the presentation of projects to best suit what the community values. However, the nature of eliciting community preferences can be difficult. One must bypass the acknowledgement of communities that multiple aspects are important to reveal actual preference formation. Measuring preferences at the individual level provides the ability to make comparisons between members of the community to determine overall consensus of preferences within the community. To make these comparisons, individual preferences are consolidated into higher-order data to assess community preferences.

The work presented in this article introduces a survey research design to characterize preferences regarding future mission attributes. This study is novel by identifying preference strength and complementing preference mapping with an analysis of open responses that participants made in motivation of their preference. This approach has the advantage that the individual preferences as well as the motives for the preference are provided by the same individual. Besides quantitative analysis, a thematic analysis is chosen to identify topics, ideas, and patterns in the survey data and to aggregate individually held community preferences into a thematic map. Thus, with a detailed, evidence-based approach, this study investigates community preference about decadal mission concepts, capabilities, and primary sciences.

The main contributions of this study are:

  1. Identify individually held preferences that form community preference in regard to NASA's next decade.

  2. Examine consensus among community subgroups concerning preferences for NASA’s next decade.

  3. Analyze survey responses from community members to determine preferred missions that are visible within the community and those that are not.

2. Methodology

The goal of this study was to examine the astrophysics community to investigate their expressed preferences. To meet the goal of this study, a large set of questions was analyzed to identify individually held preference elements within them. Figure 1 presents a summary of the study’s methodology.

Figure 1. Methodology Chart

2.1 Participants

Participants were members of the astronomy and astrophysics community (N = 322) from various backgrounds who attended the June 2018 AAS conference in Denver (N = 109) and the January 2019 AAS conference in Seattle (N = 213); 57 were excluded because of missing data or if requested. The demographic arrays of participants are presented below (Table 1). Overall, the participant pool revealed three major disciplines: academia (e.g., tenure-track and tenured professors/ research staff), students (e.g., undergraduate and graduate, and non-academia (e.g., management, data analyst etc.).

Table 1. Demographics of Participants in the June 2018, January 2019, and Combined Samples

Demographics

June 2018

January 2019

Total

Gender

% Men

61.0

58.0

59.2

% Women

38.1

40.6

39.7

% Other

-

1.4

0.9

% Missing

1.0

-

0.3

Race/Ethnicity

% Caucasian

76.2

70.3

73.9

% African American

1.0

3.3

2.6

% Asian

9.5

11.8

11.3

% Hispanic/Latino

5.7

6.1

6.1

% Native American / Alaska Native

-

0.5

0.3

% Native Hawaiian/ Pacific Islander

1.0

-

0.3

% Multiracial

1.9

6.1

4.8

% Other

-

0.9

0.6

% Missing

5.6

0.9

2.2

Based on the literature’s emphasis on the description of differing contexts, two contextual community subgroups were formed by researchers. The sorted groups of space-based mission involvement were analyzed to better understand preference differences within sub-groups of the community. The groups were designated based on responses given by participants within the demographic questionnaire. Of the community participants, 30.1% had worked on space-based missions, while the majority of 59.9% had no direct involvement in space-based missions (Table 2). Further, of those who have worked on space-based missions, 46.9% were non-academia professionals, 20.2% were academic professionals, and 21.9% were undergraduate or graduate students.

Table 2. Participants Involved in Space-based Missions

Subgroup

Response

Number of Participants

Worked on a space-based mission

Apollo, AstroSat, Bepicolombo, Cassini, Chandra, CRGO, EINSTEIN, EPOXI, EUVE, FUSE, Gaia, GALEX, Herschel, Hinode, Hipparcos, Hubble, ICON, InSight, Integral, IRAS, ISO, James Webb, K2, Kepler, LCROSS, Lucy, Mariner, Mars 2020, MER, MESSENGER, MRO, MSX, New Horizons, NICER, NuSTAR, Parker Solar Probe, Pathfinder, Planck, ROSAT, RXTE, SCORPIO, SOFIA, Spitzer, SWAS, Swift, TESS, Viking, Voyager, VSOP, XMM-Newton

96 (30.3%)

Not worked on a space-based mission

192(60.6%)

Information withheld

29 (9.1%)

2.2 Survey Development and Set Up

The study of community preferences for the coming decadal survey resembled a survey design. The development of the survey research followed a six step process. (1) Conceptualization and design of the survey; (2) Institutional Review Board (IRB) approval; (3) Execution; (4) Revision; (5) Data collection; and (6) Data monitoring and evaluation. The goal of the survey was to gather data to show the formation of preferences to better understand community preferences regarding the decadal survey. During the first step, the researchers collaborated with subject matter experts from NASA to provide preliminary input on the decadal survey to define the survey questions. This resulted in concrete and quantifiable concepts that were identified and translated into questions. A total of 37 questions made up the survey; questions were refined and revised by the research team (i.e., following steps 2, 3 and 4). The questionnaire combined scale questions (e.g.: Likert Scale) and open-ended questions to capture the preferences of the community for deciding factors of the new decadal election, specifically the perceived value added or subtracted for certain missions (e.g., flagship versus probe class missions)and the professional risk attitude of the respondents. The questionnaires enabled responses to be gathered from large numbers of AAS community members in a time and cost effective manner. Refer to Table 3 to view examples of the open-ended survey questions.

Table 3. Qualitative Questions Included in the Survey

Question

Example

1

What do you value most with regards to a mission concept for the next decadal consideration?

2

What do you think the astronomy community values the most with regard to the next decadal mission?

3

What science missions should NASA strategically invest in for the next decadal?

4

Which is more important, probe class missions or flagship class missions, and why?

5

In light of the diff. decadal concept studies, what capabilities of future astrophysics missions are desired?

6

In which ways, if any, should NASA astrophysics flagship observatories complement other missions?

2.3 Survey Administration Procedure

Participants were selected by convenience sampling; researchers carried tablets and approached members of the community during the conference and requested participation and the immediate completion of the survey. Although condition specifications were pre-determined, participants were not presented with a condition and instead the conditions were derived from the responses to a demographic question regarding the position that the participant held; participant responses were valued for real experience.

The survey was administered through the Qualtrics survey platform (https://www.qualtrics.com/), which participants accessed through either a tablet or laptop device at the AAS conference in June 2018, Denver Colorado and at the AAS conference in January 2019, Seattle Washington. At the AAS Conference in June, a poster was associated with the survey to inform participants about the research intent and was used to promote the study. Participants who were not willing to complete the questionnaire at the time were given a business card with a specific link to the survey, so that they could complete the survey on their own time. The survey being completely digitized meant that participants were first presented with a consent form that required a response to indicate consent was given for their participation before they could proceed with the survey. The survey then presented the participants with 38 closed and open-ended group preference questions. At last, participants were asked to complete a post task questionnaire that was used to gather information on demographics (age, sex, ethnicity) and other information such as past experiences (i.e. past work on missions that have flown), then they were debriefed with the option to exclude their data from the survey. All participants were given the same survey with no time limit to complete it. The total survey time was approximately 15 min. To eliminate respondent bias, a script was used to ensure technicians would provide the same information to all participants. At the AAS conference in January, researchers approached participants directly to request participants and asked whether they had participated in the survey in June 2018. Only participants that had not participated in the questionnaire in June 2018 were able to participate; eligible participants were presented with the survey. Once the survey was closed, the results were exported to SPSS. SPSS was used to analyze the preference frequencies and descriptives of the demographic information.

2.4 Thematic Analysis

Thematic analysis, a method for identifying, analyzing, and reporting themes within data was used [8]. The purpose of applying this analysis technique was to provide insight into individual preferences as themes emerge inductively from the data. This approach forms a holistic image of the collective preference of individuals and the community as a whole. After the data was collected, the analysis of the qualitative data followed an open coding process: de-contextualization, recontextualization, and identification of themes and compilation of themes to draw conclusions [9].

Two coders were trained in analyzing the data of the open-ended questions. In the process, coders had to first transcribe and read the data by breaking the data into meaningful units. Thematic codes and original text were compared to ensure selected themes were the most pertinent to the data. The themes and the units were then categorized to inform an overall thematic map (see Figure 2). Conclusions could then be drawn from the results of the previous coding stages.

3. Results

In order to examine individual preferences and consensus among community subgroups regarding the upcoming 2022-2023 decadal period, chi-square and t-test analysis were computed. Participants were asked to provide their preference score for the upcoming decadal survey, based on a 5-point scale, 1 = ‘strongly agree, and 5 = ‘strongly disagree. Individual preference scores were aggregated to the community level by analyzing preference strength in the 17 survey questions.

Chi-square analysis was run showing preferences to be statistically significant for all questions at p = .000 indicating a preference among community members of a certain score (see Table 4 for Mean, Mode, and SD). Results indicate that members of the community find both flagship and probe class mission to be of great importance. A strong preference for multi-messenger astronomy such as electromagnetic radiation, gravitational waves, cosmic rays, x-rays, etc. was also shown by community members. While the community did not display a preference for cost, results indicate mission success to be of great importance. Further, a balanced program when considering risk, capability, and cost with multiple class missions across astrophysics was preferred.

Figure 2. Thematic Map of Community Preferences for the Next Decade of NASA

In addition, the community sees value in partnering with non-NASA space agencies for the upcoming 2022-2023 decadal period. See Figure 3 for comprehensive results.

Overall, the community does not find it important to launch a decadal mission within the decade the mission is selected in the event programmatic or technical risks increase or when science goals have to be reduced. Additionally, community members do not prefer lower cost/less capable flagship missions compared to higher cost/more capable missions when the cost stands in direct proportion to the scientific capacity. Moreover, the community does not prefer less complex/ less capable flagship missions compared to a high complex/ more capable mission when technical complexity stands in direct proportion with scientific capacity.

Participants were allowed to select a neutral score indicating that an individual does not have a preference towards that particular issue. For instance, the majority of individuals displayed a neutral score on whether their personal preferences aligned with more than 80% of the astronomy community. Although the community seems to prefer high scientific return, the community displayed no strong preference for either a high risk, high cost, high scientific return or a low risk, low cost, lower scientific return mission. In addition, the community indicated no preference of whether they would recommend investing in low Technology Readiness Level technologies for flagship missions. Conclusively, the community showed no preference towards multiple probe class missions compared to a flagship mission or towards a particular flagship mission.

Expanding on the prior findings and to understand whether preferences were in consensus amongst community subgroups, the community was divided into two groups (i.e., working on a space-based mission(s) vs. not working on a space-based mission(s)). An independent t-test was deployed, indicating no significant difference regarding preferences between the two groups, p > .05 for the majority of the questions. However, significant differences between groups were found for whether a particular flagship mission is of importance; the data showed that the subgroup currently working on a space-based mission disagreed with the statement (Mean = 2.56, Std. Dev. 1.18) while those that were not working on a space-based mission indicate no preference (Mean 3.22, Std. Dev. 1.0; t(162,2) = -4.69 p < .000). This finding indicates a positive inclination of the community favoring scientific versatility, capability, and scientific return instead of favoring a single flagship mission. Overall, these results indicate that the majority of the community shows cumulative consensus among preferences. While individual preferences might vary among community members, the preferences strength provides insight into what the community desires as a whole.

Figure 3. Overview of Strength of Preferences

Table 4. Overview of Mean, Mode, and Std. Dev. of Closed-Ended Prompts

Prompt

Mean

Mode

(Score)

Std. Dev.

Flagship class missions (costing several billions of U.S. dollars, such as Hubble and Chandra) are important to me.

1.49

1

.70

Probe class missions (costing approximately 1 billion U.S. dollars) that are smaller than flagship, but larger than a medium explorer class mission, are important to me.

1.70

2

.75

I prefer multiple probe class missions to a flagship class mission

2.88

3

.99

A particular flagship class mission is important to me

2.94

3

1.10

It is important to have multi-messenger astronomy such as electromagnetic radiation, gravitational waves, cosmic rays, x-rays, etc. in the 2030s.

1.48

1

.70

The mission’s cost is important to me

2.58

3

.96

The mission’s success is important to me

1.33

1

.59

I would recommend investing in low Technology Readiness Level (TRL) technologies for flagship class missions

2.64

3

.93

I am willing to accept programmatic risks in a probe class mission

2.23

2

.78

I am willing to accept programmatic risks in a flagship class mission

2.59

2

1.02

I am willing to accept technical risks in a probe class mission

2.38

2

.99

I am willing to accept technical risks in a flagship class mission

2.99

2

1.19

Assuming cost is directly proportional to scientific capacity, it is important to have a lower cost/less capable flagship class mission than a higher cost/more capable mission

3.56

4

1.02

Assuming technical complexity is directly proportional to scientific capacity, it is important to have a less complex/less capable flagship class mission than a highly complex/more capable mission

3.40

4

.96

A high risk, high cost, high scientific return mission is more desirable than a low risk, low cost, lower scientific return mission

2.56

3

1.01

I prefer a balanced program (with respect to risk, capability, and cost) with multiple class missions across astrophysics

1.84

2

.80

It is important for a mission to partner or collaborate with non-NASA space agencies

1.99

2

.90

It is important to launch a Decadal mission within the decade the mission is selected even if science goals have to be reduced to do so

3.00

4

1.04

It is important to launch a Decadal mission within the decade the mission is selected even if technical risk is increased to do so

3.36

4

1.02

It is important to launch a Decadal mission within the decade the mission is selected, even if programmatic risk is increased to do so

3.22

4

.96

My personal preferences align with more than 80% of the astronomy community

2.77

3

.70

* All questions are statistical significance at p = .000.

Note. The mode depicts the community preference score (1 = strongly agree, 2 = agree, 3 = neutral, 4 = disagree, 5 = strongly disagree

3.1 Exploration of Preferences from Participant Open-Ended Responses

Participants were asked to provide remarks, enabling them to explain in more detail what and why they preferred certain decadal mission aspects over others (e.g., importance of a future flagship mission compared to a probe class mission). Participants were not forced to provide an explanation, so there were many participants who did not provide any response at all. Based on previous research, the nonresponses show a greater degree of neutrality than responses that held a remark, indicating that participants had no strong opinions on the prompt [10]. Thus, a total of 4,859 words were generated by participants for the eight open-ended questions contained in the survey. Emergent themes are articulated around four research questions:

Research Question 1: What does the community value with regards to a mission concept for the next decadal consideration (Questions 1 and 2 in the survey)?

Research Question 2: What science mission, capabilities, and primary science should NASA invest in? (Question 3, 6, and 8 in the survey)

Research Question 3: Which mission types are desired and how should flagship observatories complement other missions? (Question 4, 5, and 7 in the survey)

Figure 4. Word Cloud Plot of Qualitative Response in Community Preference

Note. The font size corresponds to the relative importance of each topic in the response. The most influential topics are the largest.

Quotes presented throughout the results and in Table 5 are taken directly from participant responses to allow the reader to judge the veracity of the themes that emerged. Further, an initial word frequency analysis of community answers revealed certain topics to be of greater value to the Astronomy and Astrophysics community compared to others (see Figure 4). The word cloud represents frequency of a word using size, bigger words represent more frequent words. Words such as ‘science’, ‘mission’, ‘flagship’, ‘exoplanets’ and ‘probe’ were some of the most common words discussed by the community

Table 5. Example Quotes from Community Responses

Question

Community Responses

“What do you value most with regards to a mission concept for the next decadal consideration?”

  • “Scientific versatility. The mission must not be limited to a single class of objects (e.g., exoplanets or AGNs), but must be capable of profitably observing many types of objects.”

  • “Aim big to answer fundamental questions - exoplanets, dark matter, dark energy, star/planet formation”

  • “Multi messenger astronomy”

  • “Highest scientific return”

  • “Executability on reasonable timescale”

  • “Exoplanet capability at a cost that does not cause disruption in the balance of the overall program.”

  • “Including non-NASA institutions in the designs and build”

  • “Large aperture UV optical space telescope”

  • “Complementary technology development with ground based and other spsce based missions”

  • “Exoplanets”

“What do you think the astronomy community values the most with regard to the next decadal mission?”

  • “Getting more than one mission off in a reasonable timeframe”

  • “The ability to answer the most important questions”

  • “Extra-solar planets”

  • ”increasing resolution and sensitivity”

  • “balance between ground-based and space-based missions”

  • “Multiwavelentgh”

  • “Realistic cost estimates.”

  • “The best quality data from a dependable source”

  • “Large aperture UV optical IR space telescope”

  • “WFIRST”

“What science missions should NASA strategically invest in for the next decadal period?”

  • “High energy synergies with gravitational wave astronomy”

  • “Earth observation/monitoring, astronomy/cosmology observation, robotic exploration (in order)”

  • “Origins of life, from molecular clouds to planets; signs of life on exoplanets; links between protoplanetary disks and exoplanets.”

  • “Lynx”

  • “Exoplanet surveys and high resolution spectroscopy of planetary atmospheres”

  • “WFIRST, JWST”

  • “Wide spectrum coverage”

  • “LUVOIR, OST”

  • “X-ray, high resolution imaging”

Which is more important, probe class missions or flagship class missions, and why?

  • “Probe class with current funding climate having a chance of covering more subfields”

  • Flagship; higher probability of serendipitous breakthrough science, & broader opportunities for participation

  • “Either are sufficient. If we can have multiple (which implies probe) missions advancing different areas of compelling astrophysics within a decade, it broadens the field for the largest community”

  • “I think that a flagship class mission is probably necessary for studying biological signatures, for the sensitivity issues. I would only support probe class missions if there were enough of them to cover multiple wavelengths and astronomy fields.”

  • “flagship mission for bigger scientific return and to appeal the public”

  • “They are equally important and each serve there own purpose. the probes can accept more risk and be the "work horses" while the flagships offer capabilities and open observing phase space that cannot be done by a probe”

  • “I think the most prudent approach is to have an appropriate mix for the profile of the current decade.”

  • Probe and possibly smaller (e.g. cube sats). Because more science return, more responsive to budget and schedule constraints.

  • Probe class thought less capable (generally) have greater breadth, and lower risk - which seems to balance well with public perception

What flagship class mission concept(s) is/are important to me?

  • “Lynx and XARM”

  • “Luvior”

  • “LISA and James Webb telescope”

  • “OST - uniqueness”

  • “Ones that are not just improvements over current ones, but ones that result in order-of-magnitude understanding of the universe, origins, and "is there life..."”

  • “LISA and Lynx”

  • “LUVOIR and/or Origins”

  • “luvoir, habex, ost”

“In light of the diff. decadal concept studies, what capabilities of future astrophysics mission are desired”

  • “High effective area, high resolution xray spectroscopy and wide field imaging”

  • “high resolution spectroscopy, far-ir, near-mid ir”

  • “durability; have core mission, yet ability to continue ad infinitum....like voyager, new horizon....”

  • “studying exo-planet atmospheres”

  • “polarimetry, coronagraphs”

  • “IFU, high resolution, uv”

  • “Terraforming”

  • “Spectroscopy of exoplanetary atmospheres”

  • “general astrophysics, spectroscopy, coronagraph (direct imaging), transit spectroscopy of exoplanet atmospheres.”

In which ways, if any, should NASA astrophysics flagship observatories complement other missions?”

  • “Flagships should be the general learning and discovery instruments providing opportunities to define smaller scoped follow up projects.”

  • “By doing things that can't be done from the ground - i.e., UV and X-ray observations, and high contrast imaging”

  • “I think that multi-wavelength observations from different observatories SHOULD complement each other.”

  • “Missions should, when possible, build upon one another.”

  • “More international collaborations, higher scientific return requiring risk”

  • “Complimentary non-overlapp ing wavelength and spatial resolution”

  • “They should seek to expand upon and improve existing missions”

  • “planet formation and exoplanet research are complementary for all STEM fields”

  • “Multimessenger observations of the same objects”

  • “Cover areas of interest that other missions haven’t covered yet, or cover it with better resolution/sensitivity”

  • “They should be able to be used simultaneously with other observatories to increase knowledge”

  • “There should be a span of wavelengths offered. There needs to be another UV observatory so that all wavelengths can be studied equally”

“what primary science would you like to see a future NASA astrophysics observatory address”

  • “high resolution spectroscopic follow-up of resolved stars in nearby galaxies (&gt;3Mpc).”

  • “Intergalactic medium observations to account for missing baryons”

  • “Characterizing the atmospheric composition and potentially the surface states of terrestrial exoplanets”

  • “UV spectroscopy”

  • “Extending the range in sensitivity, resolution, or wavelength coverage”

  • “Epoch of reionization, first galaxies”

  • “Early universe”

  • “Black hole missions”

  • “Very accurate radial velocities of stellar surfaces.”

  • “gravitational-wave astronomy”

3.2 Thematic Analysis Results

The thematic analysis extracted similar topics from responses by the participants. Participants were asked to answer eight open-ended questions that lead to a total of 796 responses. From the responses, topics were then classified into similar subthemes generating 9 overarching themes. Each question themes varied from their respective responses, such as scientific return, further exploration, wavelength coverage, flagships, and so forth. Some open-ended questions resulted in more themes than others due to the wide variety of responses. A description of each thematic code is presented in Table 6. The descriptions shown were used by the coders in their coding process. Keywords that were strong indicators for the codes were also included. Thematic responses were then tallied and frequency was determined. A full selection of 33 sub-themes were coded and were collapsed into 9 overarching themes. Overall, open-ended responses indicated a strong desire for ongoing mission and mission concepts that would best accommodate observational astronomy.

A chi-square test was used to identify statistically significant differences between the numbers of responses that fell into each theme in order to analyze our research questions. Overall, the frequency of responses did not vary between subgroups, p > .05 this indicated consensus among community preferences. To answer research question 1, regarding the community's values concerning mission concepts for the decadal consideration, both question one and two of the survey were considered. Overall, the frequency of responses did not vary between subgroups, p > .05 indicating consensus among community preferences.

Table 6. Relevant Themes Used for Open Coding

Theme

Sub-themes/ Sample Keywords

# Count

Ongoing missions and mission concepts

Flagship, probe, HST, JWST, WFIRST, HABEX, Lisa, LUVOIR, Lynx, OST, ngVLA

254

Observational astronomy

Gravitational wave, IR, wavelength Coverage, X-ray, multi-messenger, multi-wavelength, resolution and sensitivity

149

Science versatility, capability, and scientific return

Ground breaking research and science return, science versatility and capability

121

Planetary Science

Planetary formation, exoplanetology

48

Astrophysics

Galactic astronomy, physical cosmology, stellar astronomy, solar physics

45

Complementary and joint missions

Complementary missions to existing missions, complementary ground and space-based missions, public and non-government institute engagement, shared resources and infrastructure

37

Reasonable mission timeframe, cost, and durability

reasonable project or mission timeframe, mission cost, mission durability

22

Astrobiology

12

Spectroscopy

7

No Preference/Answer

[Blank], N/A, All are important

425

Responses to the question, “what do you value most with regards to mission concept for the Decadal consideration” showed scientific versatility, capability, and scientific return as overwhelmingly important. Chi-square test identified this difference between the number of responses that fell into each theme to be significant X2 (8, N = 133) = 268.21, p = .000 (see Figure 5).

When asked, “what do you think the astronomy community values most with regards to the next Decadal mission,” the highest number of participants indicated observational astronomy, then ongoing space-based missions and space-based mission concepts, and finally complementary and joint missions and astrobiology, X2 (7, N = 104) = 135.23, p = .000 (see Figure 6). Also, individual values concerning the next decadal mission were significantly different from community member’s perceptions of what the community values X2 (7, N = 96) = 29.02, p = .000, indicating individual values may not completely align with the values of the community.

Figure 5. Frequency of Preference about Mission Concepts for the Next Decadal Missions

Figure 6. Frequency of Individual Perception of What the Community Values Concerning the Next Decadal Missions

To answer research question 2 regarding preferences of science missions, capabilities, and primary science NASA should attend to, questions 3, 7, and 9 of the survey were considered. Responses to the question “what science missions should NASA strategically invest in for the next decadal period,” showed observational astronomy was preferred by the community, X2 (8, N = 59) = 64.71, p = .000. A similar result was found, when asked, “in light of the different decadal concept studies, what capabilities of future astrophysics missions are desired.” Responses displayed capabilities of observational astronomy such as gravitational wave, infrared, wavelength coverage, X-ray, multi-messenger, multi-wavelength, resolution and sensitivity to be preferred, X2 (4, N = 60) = 114, p = .000. Moreover, participants indicated a preference for 1) astrophysics, specifically galactic astronomy, physical cosmology, stellar astronomy, and solar physics; 2)observational astronomy; 3) planetary science particularly exoplanetology and planetary formation, when asked “what primary science would you like to see a future NASA astrophysics observatory address,X2 (4, N = 89) = 40.83, p = .000 (see Figure 7).

Figure 7. Frequency of Preferences for the Next Decadal Period

To answer research question 3 regarding preferences for mission types and flagship observatory’s complementing other missions, questions 4, 5, and 8 of the survey were considered. Results of the open ended question “which is more important, probe class missions or flagship missions, and why,” revealed that flagship missions were preferred X2 (3, N = 117) = 32.03, p = .000, expanding on findings from the closed-ended questions. Subgroups within the community displayed the same preference towards flagship class missions p > .05 implying a strong consensus among community members (Figure 8). For flagship missions (“what flagship class mission concept(s) is/are important to me”), participants displayed a preference for specific flagship missions over others, the highest response preference being towards LUVOIR, OST and Lynx, despite LUVOIR not ranking the highest overall response, X2 (6, N = 67) = 51.56, p = .000 (see Figure 9).

Figure 8. Frequency of the Importance of Flagship vs. Probe Class Missions

Figure 9. Preference of Flagship Mission Concepts in Comparison to Ongoing Missions

Note. Responses to mission concepts are displayed in blue. Responses concerning ongoing missions are displayed as a total score in gray (i.e. HST, JWST, WFIRST.)

Participant responses for why they preferred either flagship or probe class missions are displayed in Table 9. When asked, “in which ways should NASA astrophysics flagship observatories complement other missions,” participants indicated a preference for observational astronomy, complementary and joint missions, particularly complementary missions to existing missions, complementary ground and space-based missions, public and non-government institute engagement, and shared resources and infrastructure, X2(4, N = 64) = 77.25, p = .000.

Themes listed in Table 7 were used to sort results; however, for the following responses “no preference/answer” indicated that participants did provide a preference for either a flagship or probe class mission but did not specify why. The responses indicate that overall flagship missions are preferred for their potential for scientific versatility, return, and advancements in both science and technology, whereas participants who preferred probe class missions were focused more on efficiency and results of the mission. Across conditions, probe class missions were believed to have the greatest potential for high results with minimum risk, but flagship class missions indicated the greatest belief for overall scientific return. Within the cost classifier, the flagship missions were viewed as worth their high costs, whereas probe class missions were lauded for their lower costs that could run several missions for the equivalent price of flagships.

Table 7. Summary of Preferences for Flagships and Probe Class Missions

Themes

Community Preferences

FLAGSHIP MISSION

PROBE-CLASS MISSION

Scientific versatility, capability, and scientific return

24

10

Costs

4

2

Max Results, Low Risk

--

4

Most Efficient/Results

--

9

Public Appeal

7

1

No Preference/Answer

5

10

Total

40

36

4. Discussion

The purpose of this study was to examine individual preferences representative of the astronomy and astrophysics community as a whole, as well as highlight the consensus among community subgroups regarding NASA's next decade. Overall, our findings provided substantial support for our study’s aim and research questions and extended prior awareness of preferences held in the astronomy and astrophysics community in three ways. First, our findings show a general consensus among community members concerning the upcoming decadal period. Indeed, while the community concur that both flagship and probe class missions are of importance; flagship missions are the preferred choice in view of their potential for versatility, capability, return, and advancement in both science and technology. In contrast, probe class missions were believed to have the greatest potential for their high results with minimum risk and high efficiency of the mission. Flagship missions were viewed as worth their high costs, whereas probe class missions were lauded for their lower costs that could run several missions for the equivalent price of flagships. Concerning consensus for certain missions, LUVOIR, OST and Lynx were highest on the list of the 2020 Decadal Survey mission concepts. These preferred forerunner projects are those that are highly visible within the community, suggesting that others may not have received the attention warranted by the community. On a different note, the community indicated that such flagship observatories should complement other missions of observational astronomy as well as other existing missions and ground and space-based missions. Moreover, the community prefers higher involvement of public and non-governmental institutes and sets a greater emphasis on sharing resources and infrastructure for the next decadal period.

Second and most importantly, our findings paint a clear picture of community preference concerning the upcoming decadal mission. To our surprise, findings indicate individual members hold different values compared to their perception of what the community values the most about NASA’s next decade. Indeed, individual members strongly prefer science versatility, capability, and scientific return, while members perceive the community to value observational astronomy, ongoing space-based missions, and mission concepts as well as complementary and joint missions. The perception of community members might be influenced by context such as NASA’s general mission strategies and vision. Moreover, the tendency for individuals to differ in their own preferences to the perceived preferences of their community indicates that the formation of preferences in an individual relies on different values derived from contextual and personal desires [2]. However, the community showed strong consensus that NASA should strategically invest in the science and capability of ‘observational astronomy’. Community members showed particular preference for multi-messenger astronomy, gravitational-wave astronomy, wavelength astronomy, resolution and sensitivity. Moreover, the community would like to see future observatories to address the primary science of astrophysics, observational astronomy, and planetary science with a strong preference for exoplanetology.

Third, our findings suggest that the community showed strong preferences in terms of risk, capability, and cost in the upcoming decadal period. For instance, when programmatic and technical risks arise or when science goals would have to be reduced, the community preferred to wait to launch a decadal mission beyond the decade the mission was selected to ensure mission success. Moreover, the community did not indicate cost to be of importance. However, when the cost stands in direct proportion to scientific capacity, community members prefer higher cost and a more capable flagship mission in comparison to a lower cost and a less capable flagship mission. Additionally, preferences towards a highly complex and more capable flagship mission compared to a less complex and less capable mission are shown by members when technical complexity stands in direct proportion with scientific capacity. Conclusively, mission success has been shown to be of great importance to the community.

4.1 Limitations and Future Research

As with all research, there were some limitations. One of the limitations of this study was the method of participant selection. Because researchers approached potential participants by convenience sampling within a conference, there was likelihood for potential participants to decline participation. There was also a general impatience for participants who did agree. Bluntly stated, the length of the survey took time away from participants’ ability to enjoy the conference they were attending and many were eager to finish so they could return to the poster presentations and booth displays. Because of this general impatience, there is a chance that responses were hastily made by relying on instinct, which has value in lacking overthinking, but may also show a difference between responses if participants had taken more time or did not feel as if the survey was keeping them from the conference. Additionally, participants may have found it hard to justify the importance of low cost missions when space-based astronomy is inherently high-risk. Another limitation was that compared to ground-based missions, space-based astronomy is inherently high-risk wherein failure of a flagship mission is not an option as can be seen in the ongoing delays of the launch of JWST. Such nuances might have jarred participants when presented with prompts such as “it is important to have a lower cost/less capable flagship class mission than a higher cost/more capable mission.”

Aforementioned limitations could provide directions for future research. Further studies should attempt to streamline the survey to provide valuable data while still providing valid responses because of the potential impatience factor in approaching participants during a conference. Approaching participants directly is more likely to result in immediate responses because the survey must be completed in the moment rather than via email that can be pushed off; however, the desire to return to the conference meant participants were taking the survey during conference times and reducing the finite time. Future research should strive to address these issues.

5. Conclusion

The value of this study is in the identification of community preferences regarding the upcoming 2022-2023 decadal period. This study shows consensus among community subgroups regarding preferences and presents preferred missions that are visible within the community. It is critical future decadal survey projects make use of community values and preferences to support project visibility and presentation on how the upcoming decade can be realized. While the importance of reliably and honestly displaying all information cannot be understated, the study does indicate that there are different weights that should be given to the information. The community will lose interest in projects that seemingly do not hold their values, even if their preferences are represented because they are often buried in the vast amount of information presented. Addressing what each community values allows projects to hold interest and pinpoint what matters to the community while still providing a holistic presentation to the level the community will value.

Comments
0
comment
No comments here