Skip to Main Content

Aug 1

Special Notice Regarding the COVID-19 Pandemic

Roaring Return

For important updates on Lindenwood University campus safety precautions and associated information from public health officials, please visit our Roaring Return page and the CDC's COVID-19 site. Resources for faculty preparing courses for the Roaring Return can be located at www.lindenwood.edu/teaching-online-or-hybrid.

Understanding How EdD Students View Educational Research: A Qualitative Study using Domain, Taxonomic, Componential and Text Mining Analysis

Article by Chen Zong, Courtney Donovan, and Dara Marin Prais

Understanding How EdD Students View Educational Research:  A Qualitative Study using Domain, Taxonomic, Componential and Text Mining AnalysisAbstract

The purpose of this qualitative study is to explore how EdD students initially view educational research and themselves as researchers before taking their first required research course. This study used four types of qualitative data analysis methods: domain, taxonomic, componential, and text mining. The findings suggest that the EdD students are able to identity several attributes of research, but there is a dissonance on the attributes aligned with upper academic research.  The students understand the importance of research to educational practices, but do not have sufficient understanding about research methods and methodologies.  Their views of what research is are formal but their views on who does research is informal. Recommendations for EdD research course designs are offered.

Keywords: curriculum design and evaluation, graduate education, educational leadership, qualitative research

Introduction

Research skills are important for educational practitioners to improve school quality and student outcomes; thus, it is important for graduate schools of education to understand how to support practitioners’ learning of research in Doctorate of Education (EdD) programs (Kerrigan & Hayes, 2016).  The EdD is a professional doctorate in education that “prepares educators for the application of appropriate practices, for the generation of new knowledge, and for stewardship of the profession” (Perry, 2015, p. 58). Because EdD programs have a practice-focused nature, which is distinctly different from the research-focused PhD programs, the traditional coursework requirements in research methodologies for doctoral programs are questioned and critiqued by many educators who believe that EdD students should be consumers of research rather than producers of original research (Andrews & Grogan, 2005; Prestine & Malen, 2005). Although EdD course designs are grounded in a professional knowledge base, these educational practitioners need to know how to integrate practical and research knowledge (Perry, 2015). Specifically, educational practitioners need to design innovative solutions to address educational problems using practical research and applied theories in conjunction with their professional wisdom as tools for social and educational change, especially when they understand the importance of equity and social justice (Perry, 2015). Therefore, research learning and course designs for EdD programs should be tailored to better meet the specific needs of educational practitioners (Hochbein & Perry, 2013).

Research context

This qualitative study is a part of a larger program evaluation for the EdD program in the School of Education (SOE) in a public university. This program is designed for educational practitioners who are either working in education or pursuing education-related careers, such as school leaders, administrators, teachers, and educational policy-makers. As a coursework requirement, all EdD students in the SOE must take three applied research courses.  Although many of these students have had prior research or statistical coursework, 77% of our students reported coursework was over 10 years ago or never.  It is not surprising when they encounter problems or difficulties with learning and applying concepts.  Therefore, our EdD research courses have been modified over the past two years to be very application-focused and less technical-focused, with the intended result of positively impacting EdD students’ views on research and learning.

Purpose of study

The main purpose of the larger project evaluation is to better understand the perspectives, experiences, and learning needs of EdD students about research to improve the research course designs in the SOE and other similar-context universities. The larger program evaluation project consists of the repeated administration of a single survey instrument pre and post coursework. The survey instrument includes both open- and closed-ended questions for qualitative and quantitative analyses, respectively.  The pre-survey was administered to all EdD students entering the Applied Research Course #1 and the post survey at the end of the Applied Research Course #3.

The specific purpose of this qualitative study is to explore how EdD students initially view educational research before taking their first required research course in SOE and how they view themselves as researchers using the 2017 and 2018 pre-surveys’ qualitative data. Their initial view of educational research before their first class is important to understand as it informs their learning interests, goals, needs, or backgrounds related to research. A main research question was explored in this study: How do EdD students view educational research before taking their first research course?  The results of this study could be useful for informing research courses in other EdD programs.

Conceptual Framework

As a conceptual framework for this study, the definitions of research and educational research were used from two research organizations, the Office of Research Integrity (ORI) of the United States Department of Health and Human Services, and the American Educational Research Association (AERA). These definitions were decided upon before the researchers started analyzing the responses of the EdD students about their perspectives on research, researchers, and the importance of research to education. The conceptual framework is important to understand how the perspectives of the EdD students are related or different from the official definitions of educational research.

The ORI (2018) defines research as “a process to discover new knowledge” (para. 1) and indicates that research is different from other forms of discovering knowledge because it uses scientific methods in a systematic process, including observing, creating a hypothesis, testing the hypothesis, and then examining the results of these tests. Researchers can find the most important factors to a topic or problem studied through the process of a research study, including interpreting the information he/she collects and making sound conclusions about the results (ORI, 2018).  The ORI (2018) also points out that the value of research depends on the quality of the research design and process, which needs to be developed by experienced researchers who have strong research skills. In addition, the researchers should be sure that the results of the research are real and useful to other scientists, researchers, or any people related to the research study (ORI, 2018).

Educational research is a type of human subject research focused on education-related problems, and is defined by the American Educational Research Association (AERA) (2018) as “the scientific field of study that examines education and learning processes and the human attributes, interactions, organizations, and institutions that shape educational outcomes” (para. 1). The main purpose of educational research is describing, understanding, and explaining “how learning takes place throughout a person’s life and how formal and informal contexts of education affect all forms of learning” (AERA, 2018, para. 1). Educational researchers can use different types of rigorous methods, appropriate to the research questions, and also develop new tools and methods to better address the emerging educational research problems (AERA, 2018).

Literature Review

There appears to be discontinuity between the broad understanding of the importance of education research, and the perceptions and understandings of practitioners as they learn and use research within their specific fields. Only a few previous research studies investigated the EdD students’ perspectives on educational research, but these studies were conducted when they had already taken some research courses (e.g., Buss & Avery, 2017; Lindsay, Kerawalla, & Floyd, 2017).  Some other research studies on EdD students examine the learning experiences of EdD students and faculty members in research courses, but not the concepts or methods underpinning research itself, and instead provide useful suggestions on research course designs and teaching strategies for EdD programs (e.g., Buss, Zambo, Zambo, Perry, & Williams, 2017; Kerrigan & Hayes, 2016).  Such studies are helpful in understand other EdD programs with differing contexts, backgrounds, and course designs, especially in regard to their approach towards pedagogical and practical problems.

Similar to the purpose of this study, the Kerrigan and Hayes’s (2016) study investigated EdD students’ interests in conducting research for the purpose of improving research course offerings for their EdD program. They found that EdD students’ average level of interest in research ranged from middle to high with no change indicated after completing required research courses, nor did they find any significant relationship between interests in conducting research with (a) self-efficacy or confidence and (b) prior research experiences (Kerrigan & Hayes, 2016). EdD students’ interests in and values of conducting research may be an important factor indicative of whether they will value and use research for their practical work in education. Therefore, Kerrigan and Hayes (2016) suggested that EdD programs should teach students explicitly how to apply research skills to data-driven accountability or other school improvement expectations in their workplaces, which could positively influence their interests in learning and conducting research.

Other studies also provided suggestions for EdD research course designs and teaching strategies based on their research of EdD students’ learning experiences in research methodologies. For example, Buss and Avery (2017) suggested that EdD programs should offer “educationally relevant, work-related research skills” (p. 297). Similarly, Buss et al. (2017) suggested that the research courses of EdD programs should teach practitioner-focused research skills that can be used to improve their practices in education; and thus, the research courses should blend professional knowledge and research knowledge to solve practical problems in education. Lindsay et al. (2017) developed a nine-element framework for the teaching and learning activities in EdD programs to meet the students’ learning needs in research. The nine elements include: (1) developing research and study skills, (2) blending theory and practice, (3) building supportive relationships between EdD students and supervisors, (4) reflecting on theory and practice, (5) building the resilience of handling the problems in research processes, (6) developing the identity of EdD students as “researching practitioners,” (7) engaging with new opportunities in research, (8) disseminating research, and (9) making a difference in their own work and research settings. Based on the nine-element framework, Lindsay et al. (2017) suggest that it is important for the EdD program designers to consider how students could develop their learning as researching practitioners, especially as it pertains to the challenges of developing a research mindset. As a result, academic writing and the skills of research reporting and sharing should be also considered in the teaching and learning activities of EdD research courses (Lindsay et al., 2017).

The element missing in this literature conversation is in regard to how EdD students view research in and of itself. The argument can be made that understanding how EdD students view research is a foundational component of effectively designing EdD course work to instill and promote the value of research within their educational practice. Moreover, understanding their views prior to them beginning their research-focused course work is paramount if graduate schools seek to change or enhance their thinking.  Only then can we tailor course sequences to meet both program and practitioner goals.

Method

This study used a phenomenological approach to evaluate student responses. Phenomenology is a type of research design that examines the “meaning of experiences of a phenomenon for several individuals” (McCaslin & Scott, 2003, p. 449). The goal of a phenomenological study is to understand “meaningful concrete relations implicit in the original description of experience in the context of a particular situation” (Moustakas, 1994, p. 14). In this study, the EdD students’ perspectives on research, researchers, and the importance of research to education are investigated and considered the “phenomenon.”

Participants and sampling

There were 114 EdD students who participated in this study. The participants were selected based on a census sample of students in two cohorts. Sixty-four EdD students participated in the 2017 pre-survey and 50 EdD students participated in the 2018 pre-survey. 

Data collection

The survey used for both the 2017 and 2018 pre-survey data collection are the same, consisting of both closed- and open-ended questions for quantitative and qualitative analysis purposes respectively, for the larger program evaluation project. To answer the research question of this qualitative study, only the three open-ended survey questions were used as the qualitative instrument in this study: 1) What is research? 2) Why is research important to education? and 3) Who does research?  A Qualtrics survey link was emailed to students from their course professor and posted in the course online shell with the request to complete it before the first class of their first research course.

Ethical considerations 

Data were initially collected for course use, teaching modifications, and simple evaluation of course goals.  As the data were pre-collected for the course improvement purpose, there was no consent form needed.  Original data were stored in a private password protected folder, and shared only with the Principal Investigator and the course instructors. The study and data had no impact on the student's grade, coursework, or degree.  Participants were de-identified for the present research.  Once a research frame was overlaid on the evaluation data, the researcher sought and received approval by the university internal review board. 

Data analysis

Utilizing more than one type of qualitative data analysis method is important in order to understand a phenomenon more fully in a research study, which is also known as “data analysis triangulation” or “methodological triangulation” (Leech & Onwuegbuzie, 2007, p. 575). This study used four different types of qualitative data analysis methods selected from the 18 qualitative data analysis techniques presented by Leech and Onwuegbuzie (2007). The four methods selected for this study include: (a) domain analysis, (b) taxonomic analysis, (c) componential analysis, and (d) text mining analysis. All four methods were used to analyze the EdD students’ responses to each of the three open-ended survey questions in this study.

Before conducting the data analyses, all three researchers were trained with the qualitative data analysis methods used in this study. Domain, taxonomic, and componential analysis are firstly conducted on each of the three open-ended survey questions by each of the three researchers. In order to assess the interrater reliability, or “investigator triangulation” (Leech & Onwuegbuzie, 2007, p. 575), the initial data analysis results of domain, taxonomic, and componential analysis were discussed and edited in three meeting sessions for an overall agreement amongst the researchers. The procedures of data analysis triangulation, as well as investigator triangulation could increase the rigor and trustworthiness of these qualitative data analysis findings (Leech & Onwuegbuzie, 2007).

Text mining analysis was conducted as the last step of data analysis in this study. Because this type of qualitative data analysis could be conducted completely and automatically by a computer software tool, the results from text mining analysis were compared with the results from the domain, taxonomic, and componential analysis by each researcher. The intended purpose was to identify any similarities, important differences, or interesting findings from these comparisons.

The four types of qualitative data analysis methods and the steps of conducting each analysis are described as following:

Domain analysis.  Data were first analyzed using the domain analysis method (Spradley, 1979), which is a qualitative data analysis strategy that represents a search for the larger units of cultural knowledge (Leech & Onwuegbuzie, 2007). According to Leech and Onwuegbuzie (2007), semantic relationships are utilized in domain analysis to help uncover domains, which are created from cover terms (concepts; Y), included terms (referents; X), and a semantic relationship between the cover term (Y) and the included terms (X).

Spradley (1979) proposed nine types of widely-used semantic relationships in domain analysis: strict inclusion, spatial, cause-effect, rationale, location for action, function, means-end, sequence, and attribution. The present study followed Spradley’s (1979) six-step process of domain analysis: (1) selecting a single semantic relationship, (2) preparing a domain analysis worksheet, (3) selecting a sample of informant statements, (4) searching for possible cover terms and included terms that appropriately fit the semantic relationship, (5) formulating structural questions for each domain, and (6) making a list of all hypothesized domains.

Taxonomic analysis.  As the second step, data were analyzed using the taxonomic analysis method (Spradley, 1979) which is a qualitative data analysis method that (a) includes a set of categories organized on the basis of a single semantic relationship, (b) shows the relationships among all the folk terms in a domain, and (c) reveals subsets of folk terms and the way these subsets are related to the domain as a whole (Leech & Onwuegbuzie, 2007; Spradley, 1979). Spradley’s (1979) eight-step taxonomic analysis method was used in this study: (1) select a domain for taxonomic analysis; (2) identify the appropriate substitution frame for analysis; (3) search for possible subsets among the included terms; (4) search for larger, more inclusive domains that might include a subset within the one being analyzing; (5) construct a tentative taxonomy; (6) formulate structural questions to verify taxonomic relationships and elicit new terms; (7) conduct additional structural interviews; and (8) construct a completed taxonomy. The additional interviews, indicated in Step 7, were not conducted in this study.

Componential analysis. Data were analyzed using the componential analysis method (Spradley, 1979) after finished domain and taxonomic analyses. Componential analysis is defined as “the systematic search for attributes (components of meaning) associated with cultural symbols” (Spradley, 1979, p. 174). This type of qualitative data analysis can be conducted after domains are created for two main purposes: (a) uncover relationships between words, and (b) discover the differences between the subcomponents of domains (Leech & Onwuegbuzie, 2007). This study followed Spradley’s (1979) eight-step process of componential analysis: (1) select a contrast set for analysis, (2) inventory all contrasts previously discovered, (3) prepare a paradigm worksheet, (4) identify dimensions of contrast which have binary values, (5) combine closely related dimensions of contrast into ones that have multiple values, (6) prepare contrast questions to elicit missing attributes and new dimensions of contrast, (7) conduct an interview to elicit needed data, and (8) prepare a completed paradigm.

Text mining analysis. As the last step of data analysis, the data were further analyzed using the text mining analysis method (Lee, Cheng, & Zeleke, 2014; Leech & Onwuegbuzie, 2008). Text mining is a type of qualitative analysis method for “analyzing naturally occurring text in order to discover and capture semantic information” (Leech & Onwuegbuzie, 2008, p. 602). The format of qualitative data in text mining can be either documents (Leech & Onwuegbuzie, 2008) or written responses to survey questions (Lee et al., 2014). Text mining is useful for qualitative research projects with a large sample size, because it allows researchers to identify themes by automatically analyzing the words in the text, especially when text mining functions are employed using specialized software such as NVivo, SAS and SPSS (Lee et al., 2014; Leech & Onwuegbuzie, 2008).

This study followed the 4-step process of text mining as indicated Lee et al.’s (2014) study: (1) preparing data: review and clean each written responses (e.g., checking spelling errors, replacing abbreviations); (2) performing “parsing” process: parses the documents into “words” or “terms” prior to the filtering and pattern identification steps; (3) conducting “filtering” process: reduces the number of terms by eliminating unwanted terms and filtering documents; and (4) conducting text pattern analysis. NVivo 12 Pro was used in this study.

First, the data were reviewed and cleaned by checking the spellings and abbreviations in all of the 114 EdD students’ responses. The spelling errors were identified and edited to the correct spellings. The title of the data set and the respondent ID numbers were all removed from the data file. Only the relevant data were kept in the data set. The cleaned data file was imported to NVivo 12 Pro. Second, the data were parsed into “words” or “terms” before the “filtering” process. NVivo’s default function for conducting a text mining analysis automatically filters out irrelevant or nuisance words, such as: ‘I’, ‘You’, ‘a’, and ‘the,’ etc. Third, the unwanted terms or words were automatically filtered out and eliminated to reduce the number of terms using NVivo 12 Pro’s “Query Wizard” function. Last, text pattern analysis was conducted using the function of “Query Wizard – Identify frequently occurring terms in content” in NVivo 12 Pro. Specifically, the analysis was set to display the 1000 most frequently used words with a minimum word length of 3 letters, with synonyms being included in the text mining analysis (e.g., “talk” and “speak”).

Results

Question 1: What is research? 

Domain analysis. Based on the syntactic structure of the first item, “what is research?”, three semantic relationships were considered for domain analysis: strict inclusion, means-end, and attribution. Although only one relationship is typically used when conducting domain analysis, the three possible relationships were due to the fact that each respondent included qualifiers in their written responses. After initially coding for the type of relationship present within the responses, the attribution semantic relationship was determined to apply to 84.4% of the responses, addressing 52% of the responses that could be analyzed through the strict inclusion relationship, and 68% of the responses that could be analyzed through the means end relationship. Therefore, to conduct the complete domain analysis, the attribution semantic relationship, “X is an attribute of Y”, was utilized. For this question, “X” consisted of the 15 possible included terms, and Y consisted of the cover term, “research” (see Table 1).

Table 1 - Domain Analysis: Question 1 “What is research?”

 

Taxonomic analysis. Further analysis of the responses resulted in the development of three taxonomic groups (see Figure 1) in which the included terms could be adequately grouped and consisted of attributes pertaining to the overall framework for research, means of conducting research, and the ends or results of research. Responses indicative of the framework category included phrases such as structured inquiry, and words such as systematic, scientific, procedure, and/or process. One response exemplifies this category, “Research is a systematic process to answer a question about what works, doesn't work, or might work, given a set of conditions.” Responses indicative of the means of conducting research category were consistent with this response of “recording observations, understandings from readings, and collecting data in an organized way,” and included other terms/phrases, such as: explore, inquiry, discovery, investigation, and seeking information. The ends or results of research category consisted of responses that included terms such as: synthesis, data analysis, reach a conclusion, answer a question, and making decisions.

Figure 1. Taxonomic Analysis: Question 1 “What is research?”

Componential analysis. Using componential analysis, each of the included terms were evaluated based on two dimensions of contrast. In light of the fact that all respondents at the time of the questionnaire administration were EdD students, the dimensions of contrast consisted of whether the identified attributes are indicative of upper academic research and/or one’s daily educational practice, see Table 2.

Table 2 - Componential Analysis: Question 1 “What is research?”

Text mining analysis. Text mining confirmed these findings in regard to which terms were combined to develop the overall included terms in the domain analysis. The taxonomic analysis was also supported by the text mining findings. In counting the total number of terms included in the taxonomic analysis, 34% of all terms were evident in the Framework category, 31% of all terms were evident in the Means category, and 35% of all terms were evident in the Ends/Results category. Regarding the frequency of terms used, there was a higher frequency of terms used in the Ends/Results category as compared to the Means category, but with similar representation amongst all three.

Question 2: Why is research important to education? 

Domain analysis. Using domain analysis, a semantic relationship of “rationale” was identified for the second question of “Why is research important to education?” to investigate the reasons for doing or learning educational research to the EdD students. Educational research was identified as the cover term (Y), and 14 possible included terms (X) were identified including: educational practices, students’ learning experiences, educational equity, educational changes, educational policy, educational effectiveness, decision-making, evidence-based understanding, finding causes and effects, learning from other research, rigorous methodologies, accurate assessment or measurement, applying theories into practices, and credibility and validity (Table 3). According to the definition of rationale relationship in domain analysis (Spradley, 1979), each of the 14 included terms (X) is a reason for doing educational research (cover term, Y).

Table 3 - Domain Analysis: Question 2 “Why is research important to education?”

Among these 14 reasons, to understand and improve educational practices, students’ learning experiences, and educational equity are the three most important reasons for doing research to the EdD students. For example, one respondent wrote, “Research is important in education in order to put best practices in place that will lead to maximized academic achievement for all students.  Research is also helpful in replicating ‘what works.’” Another noted, “Education requires research to best support instructional strategies, teacher effectiveness and accountability within student output (i.e. assessments) to show if schools are showing growth and supporting classrooms to determine how to best support students’ needs in diverse situations.”

Furthermore, the need for an evidence-based understanding of educational practices using data and information is another important reason for doing educational research. A participant wrote, “Research provides a basis for which evidence-based practices are essential to our current practice to benefit the learner and improve instructional practice.” Another wrote, “Research is important to education so that decisions related to education and student learning are informed by data and human experience so that we are not constantly re-inventing the wheel and taking guesses at best practices.”

Taxonomic analysis. Using the domain analysis results from above, the following taxonomic analysis was conducted to investigate how the EdD students’ reasons for doing research were grouped. The substitution frame (i.e., “is a reason for doing”) is presented in Figure 2. Two main subsets of the 14 included terms, the reasons for doing research, from the domain analysis were identified: (a) to understand/improve education (research contents) and (b) the strengths/benefits of research (research methods). From the perspectives on research contents, the EdD students believe that research is important for them to understand and improve education, such as educational practices, equity, changes, policy, effectiveness, student learning experiences, and decision-making. From the other side of perspectives on research methods or the strengths or benefits of research, the EdD students believe that the needs of evidence, data, rigorous methodologies, accurate measurement and assessment tools, credibility and validity, and learning from other research studies are the other most important reasons for doing educational research.

Figure 2 - Taxonomic Analysis: Question 2 “Why is research important to education?”

Componential analysis. As shown in Table 4, the 14 included terms identified through domain analysis were utilized as the contrast set. Three dimensions of contrast were formulated through information collected from the participants and reflection of the researchers to understand how the EdD program’s courses are supporting the EdD students’ reasons or expectations for learning and doing research. The three dimensions of contrast are: (a) Are the research courses supporting this? (b) Are the content courses supporting this? and (c) Are the practices or work supporting this?

As shown in Table 4, the three aspects of “rigorous methodologies” “accurate assessment or measurement tools” and “credibility and validity” are unlike the other aspects in that they could only be learned from the research courses. The aspects of “decision-making” and “applying theories into practices” might be learned from all of the three dimensions, but none of them are supporting these two aspects as a necessary content.

Table 4 - Componential Analysis: Question 2 “Why is research important to education?”

Text mining analysis. Text mining analysis using the NVivo results confirmed the researchers’ findings on Question 2. The most frequently used five words identified in Question 2 were: education (107 times), research (69 times), learning (65 times), practices (59 times), and best (42 times). These five most frequently used words confirm the findings that the most important reasons for doing educational research to the EdD students include: (a) to understand what is the best educational practice, and (b) to improve students’ learning experiences. The terms “evidence” and “data” were used 15 and 24 times respectively.  Evidence included references to demonstrating, grounded, grounds, prove, show, showing, tells.  Data included references to data, inform, information, informational, informed, and informs. These two terms also confirmed the “evidence-based understanding” is another important reason for the EdD students to learn and conduct educational research.

Question 3: Who does research?

Domain analysis. Using domain analysis, the following terms and relationships were identified: Everyone, Anyone, University/professor/faculty, Students (implied in higher education), Teachers (implied K-12), Researchers, Practitioners, Scientists, Educators, Administrators, Individuals, People, Community/Interest groups, Social scientists, Policy makers, Teams, Parent/families (Table 5).  It is important to note the difference in the words “Everyone” or “Anyone.”  Coders considered collapsing these, but realized that responses listing “Anyone” were adding a choice alongside it.  Meaning anyone can do research, but not everyone does research.  Those responses that included the word “Everyone” explained a view that all people naturally do research.

Table 5 - Domain Analysis: Question 3 “Who does research?”

Taxonomic Analysis. Due to the informal nature of the anyone/everyone view compared to other responses that were more formal, a second pass of the data coded for formal verses informal responses.  Informal responses were ones where the nature of research was informal such as curiosity, discovery, and accessing new information/understanding.  Here are a few example responses: “Research can be done by anyone - teachers, families, students, administrators - or anyone who is committed to inquiring into the why and how of a topic.”  “Anyone who wants to increase their knowledge can do research.”  “Everyone if you think about the things we do in everyday life!”  “Anybody seeking to learn, grow, or improve the community in which they participate.” 

Formal responses discussed more traditional forms of research that included or implied a methodology or process.  For example: “Those who have studied research methodologies and dedicate their practice to the act of research.”  “There is a difference between general research and formal academic research (e.g., quantitative and qualitative research).  Everyone can learn about various types of research, how to critically analyze data, and ultimately utilize the results to initiate positive change.”  “We are all scientific thinkers.  Kids do it, adults, etc.  It is just a question of how formal, and how scientific the research is.”  “More ‘sophisticated’ forms of research are conducted by students and professionals.” The taxonomic analysis shows this relationship in Figure 3.

Figure 3 - Taxonomic Analysis: Question 3 “Who does research?”

Componential Analysis. Additionally, as some responses discussed this explicitly while others were more implicit, we considered varying dimensions of responses.  64% were informal references with 36% being formal references.  Nine of the informal references were explicit and 82 had implicit references.  On the other hand, 13 explicitly referenced formal research and 28 implicitly described formal research.  What was interesting is that 20% of the informal responses included a justification statement explaining a process or setting parameters around the informal nature described.   For example, “Anyone can do research as long as the information is substantiated with empirical data, and not just assumed because some research takes time.”  “Anyone can do research, but there is a process or procedure that is standard.”  “Anyone can do research if it's conducted the right way.”

The Table 6 examines which entities were included in statements that were informal and formal responses that were explicit or implicit about the idea and for those with a justification statement in the response.

Table 6 - Componential Analysis: Question 3 “Who does research?”

Text Mining Analysis. Text mining analysis confirmed these findings.  The term everyone and anyone were used 35 and 34 times respectively, and are the most frequently used words in the data. The term of student was used 27 times and included references to scholar, scholars, and students.  The term educators was used 19 times and included references to schools.  The term teachers was used 12 times and was the 6th most frequent entity listed.  The terms university, professors, scientists, individuals, and practitioner were also frequently used in the data.

Discussion

This study analyzed 114 EdD students’ responses to understand how students view the concept of research before starting an EdD research course sequence.  Three open ended survey questions were examined using domain, taxonomic, componential, and text mining qualitative analyses.

The results show that the EdD students know the basic concepts and strengths of educational research before taking their first research course, such as examining and understanding education and learning processes (AERA, 2018) and the validity and scientific methods of research (ORI, 2018). Like many previous research studies (e.g., Buss & Avery, 2017; Buss et al., 2017; Kerrigan & Hayes, 2016; Lindsay et al., 2017; Perry, 2015), this study also found that educational practice is the most important consideration for the EdD students to view the value of educational research. Although our data analyses were not focused on EdD students’ interests in learning and conducting research, the results suggest that the EdD students have a high level of interest in learning and conducting research because they expressed very positive perceptions on the importance and value of research to education and their practical work in education. This finding can be supported by Kerrigan and Hayes’s (2016) study that EdD students have middle to high level of interests over time from the beginning of the first research course to the last one.

Question 1: What is research?

Based on the analysis of the responses to question 1, “What is research?”, EdD students are able to identity several attributes of research that can be combined into broad categories consisting of the framework of research, means to accomplish research, and intended ends or results. When these attributes are aligned with upper academic research and the realities of educational practitioners, there is clear dissonance. While most of these attributes may be indicative of academic research, not all of them are required in the daily practice of an educator. These findings are interpreted in conjunction with the fact that all respondents were EdD majors, and therefore their responses indicate an academically minded understanding of how they view research.

Question 2: Why is research important to education?

Similar to the results of question one, it is evident that the EdD students already knew why it is important to learn and use research in education.  Their reasons could be grouped into two categories: research contents and research methods. Some EdD students, who might have already had some knowledge or experiences of research from their work, know the strengths and benefits of research as a method or tool to understand and improve educational practices. On the other hand, many others only had a basic knowledge about the practical use of research. and although they know that research is important to education, they might not fully understand how scientific and rigorous research is conducted, nor why research results could be valid or reliable for practical uses. These findings show us that the EdD students positively view the importance of research to educational practices before entering their first doctoral research course, but do not have sufficient understanding about research methods and methodologies. The value of a research depends on the quality of the research design and process (ORI, 2018); therefore, the EdD students should not only be experts in their own fields of education, but also need to equip strong research knowledge and skills in order to conduct high quality research.

Question 3: Who does research?

Considering that EdD students are all practitioners studying to earn a doctorate to deepen their knowledge and further their careers, it is not surprising how informal their views are on who does research. In fact, 64% of responses listed anyone or everyone as an entity who does research.  Formal responses included lists of entities from higher education, such as faculty and graduate students, those from K-12, such as teachers, administrators, and practitioners, and those from traditional fields, such as scientists.  Of the informal responses, 20% included a justification like “as long as” alongside the response to clarify there is a process or correct way research is conducted.  This was an exciting find for our program as we take the stance that research should be applied and tested by practitioners in the field, but that applied research needs to follow rigorous methodologies. 

Implications

The findings of this study can be used for graduate schools of education to improve the course designs and teaching of research courses for EdD programs. Like many other research studies (e.g., Buss & Avery, 2017; Buss et al., 2017; Kerrigan & Hayes, 2016; Lindsay et al., 2017; Perry, 2015), this study suggests that the EdD research courses should be practice-focused and applied. However, our study has some additional and different recommendations for EdD programs and their research course designs, especially for the first research course:

  • Teach EdD students how to conduct high quality research with strong research knowledge, skills, and tools to achieve their goals of improving educational practices. This must begin with a conversation on the “anyone can do research” verses “everyone does research” finding.  The students need to see research inquiry in both daily practice and in more methodological process driven research.  More so, they must understand when each type of inquiry is appropriate.  
  • Include more currently emerging or non-traditional research methods, methodologies, and techniques for conducting research about educational equity, for example, critical race methodologies (Solórzano & Yosso, 2009), social design experimental research (Gutiérrez & Vossoughi, 2010), and improvement science (Park, Hironaka, Carver, & Nordstrum, 2013).
  • Implicitly teach EdD students how to apply research into practice and guide their decision-making. Ideally this learning would be guided and practiced before their dissertation work.
  • Account for and build upon on the knowledgebase that was evident in so many of the responses. Meaning, address their prior knowledge and strengths as a practitioner first in order to build their confidence and then their knowledge in research. 

It was obvious to the coders how important and valuable the EdD students viewed research.  Many times these doctoral students are viewed as “less than” a PhD student since they tend to be in accelerated programs focused on applied research, instead of generalizable research.  We hope this study adds their voices to show how much value and excitement they show regarding research long before they have even taken a graduate research course.

Limitations and Recommendations

As the initial purpose of the data that used in this study was only for the internal program and course evaluation, there are some limitations in this study: the sample size was small, and the online survey open-ended question was the only format of data collection. In future research studies, researchers could use a larger sample size in multiple higher education institutions with multiple data collection (e.g., interviews, focus groups, surveys, etc.) and both qualitative and quantitative data analysis methods to explore more specific factors and stories related to EdD students’ research learning experiences.

References

Andrews, R., & Grogan, M. (2005). Form should follow function: Removing the EdD dissertation from the PhD straight jacket. UCEA Review, 47(2), 10–13.

American Educational Research Association. (2018, April 7). What is education research? Retrieved from http://www.aera.net/About-AERA/What-is-Education-Research

Buss, R. R., & Avery, A. (2017). Research becomes you: Cultivating EdD students’ identities as educational leaders and researchers and a “Learning by doing” meta-study. Journal of Research on Leadership Education, 12(3), 273-301. doi: 10.1177/1942775116674463

Buss, R. R., Zambo, R., Zambo, D., Perry, J. A., & Williams, T. R. (2017). Faculty members' responses to implementing re-envisioned EdD programs. Studies in Higher Education, 42(9), 1624-1640. doi:10.1080/03075079.2015.1113951

Gutiérrez, K. D., & Vossoughi, S. (2010). Lifting off the ground to return anew: Mediated praxis, transformative learning, and social design experiments. Journal of Teacher Education, 61(1-2), 100-117. doi:10.1177/0022487109347877

Hochbein, C., & Perry, J. A. (2013). The role of research in the professional doctorate. Planning and Changing, 44(3/4), 181-195.

Kerrigan, M. R., & Hayes, K. W. (2016). EdD students’ self-efficacy and interest in conducting research. International Journal of Doctoral Studies, 11, 147-162.

Lee, C., Cheng, C., & Zeleke, A. (2014, July). Can text mining technique be used as an alternative tool for qualitative research in education? Paper presented at 15th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD), Las Vegas. doi:10.1109/SNPD.2014.6888691

Leech, N. L., & Onwuegbuzie, A. J. (2007). An array of qualitative data analysis tools: A call for data analysis triangulation. School Psychology Quarterly, 22(4), 557-584. doi: 10.1037/1045-3830.22.4.557

Leech, N. L., & Onwuegbuzie, A. J. (2008). Qualitative data analysis: A compendium of techniques and a framework for selection for school psychology research and beyond. School Psychology Quarterly, 23(4), 587-604. doi:10.1037/1045-3830.23.4.587

Lindsay, H., Kerawalla, L., & Floyd, A. (2017). Supporting researching professionals: EdD students’ perceptions of their development needs. Studies in Higher Education, 1-15. doi: 10.1080/03075079.2017.1326025

McCaslin, M. L., & Scott, K. W. (2003). The five-question method for framing a qualitative research study. The Qualitative Report, 8(3), 447-461.

Moustakas, C. (1994). Phenomenological research methods. Thousand Oaks, CA: Sage.

Office of Research Integrity. (2018, April 7). Module 1: Introduction: What is research? Retrieved from https://ori.hhs.gov/content/module-1-introduction-what-research 

Park, S., Hironaka, S., Carver, P., & Nordstrum, L. (2013). Continuous improvement in education. Carnegie Foundation for the Advancement of Teaching. Retrieved from https://www.carnegiefoundation.org/wp-content/uploads/2014/09/carnegie-foundation_continuous-improvement_2013.05.pdf

Perry, J. A. (2015). The Carnegie project on the education doctorate. Change: The Magazine of Higher Learning, 47(3), 56-61. doi: 10.1080/00091383.2015.1040712

Prestine, N., & Malen, B. (2005). The case for revitalizing the dissertation. UCEA Review, 47(2), 7–9.

Spradley, J. P. (1979). The ethnographic interview. New York, NY: Holt, Rinehart and Winston.

Solórzano, D. G., & Yosso, T. J. (2009). Critical race methodology: Counter-storytelling as an analytical framework for educational research. In E. Taylor, D. Gillborn, & G. Ladson-Billings (Eds.), Foundations of critical race theory in education. New York, NY: Routledge SAGE.

Biographical Statement

Chen Zong is a PhD candidate who studies Research, Assessment and Evaluation at CU Denver in the School of Education and Human Development. Chen’s research interests include research course teaching and learning in higher education and educational large-scale data analysis for policy and practice improvement.

Courtney Donovan is a Clinical Assistant Professor at CU Denver in the School of Education and Human Development.  She teaches graduate level research methods, qualitative analysis, and statistical modeling.  Courtney's research interests revolve around the research-to-practice gap as she believes educational research should directly work to improve and impact practitioners. 

Dara Marin Prais is a doctoral student at CU Denver, studying Research, Assessment and Evaluation, in the School of Education and Human Development. As a teacher of color, and former K-12 general and special educator, Dara’s research interests include educational equity and diversity, with a specific focus related to supporting students and teachers from all backgrounds.

Lindenwood University
209 S. Kingshighway
St. Charles, MO 63301