An essential part of being an effective teacher is keeping your knowledge up-to-date, and one of the best ways to do this is to read and think about research on teaching and learning on a regular basis. Taking deep dives into peer-reviewed published research on teaching and learning will give depth to your classroom instruction, immerse you in a world of ideas never considered before, and stir up questions that deserve investigation.
Becoming familiar with education research is especially important for instructors who apply flipped learning, because the landscape is evolving quickly. Almost 200 peer-reviewed research articles on flipped learning were published in 2017 alone; over half of all research on flipped learning has been published in the last 18 months, and that body of research is currently doubling every 16 months.
That rapid growth is exciting, but this explosive expansion also makes it hard to keep up with the research. Most instructors have precious little time for the torrent of publications gushing out every day, and many have limited experience with reading this kind of research. Those limitations make it difficult to really understand what the research is saying, and to properly apply its findings in our teaching.
In this article, I will share some tips for where to find research, how to decide what to read, how to read it, and then how to put it into practice for all of us busy instructors who struggle to find the time or direction to dive into the sea of research.
What we mean by “research” and where to find it
Research is any sort of systematic inquiry that aims to answer questions, to form new theories, or to analyze existing knowledge about a topic. We do not mean published works like op/ed pieces, blog posts, and trade publication articles. These are useful in their own ways, but a true research study – where the authors pose a question or a problem, situate it in the context of what’s already known or believed, and then investigate that question or problem in a rigorous way so that we can trust the results – has a special richness of information to it that, if we are willing to invest the time and effort, can lead to enormous payoffs in our teaching.
We also usually consider only research that is peer-reviewed, meaning that before it gets published, the study has been vetted and approved by a group of knowledgeable peers who have reviewed the study (often quite rigorously) and who vouch for its validity and significance. Also, we usually only consider works that have been published in a reputable print or online venue, like an academic journal. Many non-peer-reviewed publications can be very helpful; conversely, sometimes peer review fails to rescue a bad study from seeing the light of day. But going through a rigorous peer review process adds a layer of trust that the author’s writing is clear and unbiased, the methods used are sound, and the conclusions are valid.
It is quite easy to tap into a torrent of peer-reviewed research on flipped learning. Two especially useful free tools that you can use to find research are:
Educational Resources Information Center (https://eric.ed.gov) ERIC is a free database maintained by the United States Department of Education. It provides advanced tools for narrowing article searches, for example by looking only for peer-reviewed publications, articles with certain words or phrases in the abstract of the article, or a combination of these.
Google Scholar (https://scholar.google.com) This dedicated Google search works just like “regular” Google, but its results are limited to (usually) peer-reviewed publications. Like any Google search, you must be specific and limited in your search query. For example, at the time of this writing, entering flipped learning into Google Scholar yields about 119,000 results. Putting quotes around “flipped learning” (so the search is forced to look for that particular two-word phrase) cuts the hits down to just over 9,000. You can use tools within Google Scholar to narrow this search down further, such as limiting the year of publication. If you are looking for a specific article or works by a specific author, you should be able to locate them easily.
Unfortunately, there is a caveat to both ERIC and Google Scholar. Many research journals keep their articles behind a paywall, so when you find an article with one of these tools, you might not be able to access it. If you work or study at a college or university, then your institution probably has a subscription to some of these journals, or you can order them via interlibrary loan. If not, you might still be able to find another copy of it by doing a web search.
While more and more journals are becoming open-access, most still require exorbitant subscriptions to access their articles. The best tactic if you are paywalled out of an article is to contact a colleague or friend at a university who can access it for you.
How to decide what to read (and what not to read)
A typical search for flipped learning articles will yield far more results than a normal person can process in their entire lifetime.
Probably the simplest way to filter your research is to read the abstracts. These short executive summaries (usually 100-200 words) provide the questions, methods, and results of an article in a nutshell. You can utilize the abstract like you would use a product summary on Amazon or a movie trailer: to decide the article deserves the time needed to read and process it. Sometimes you will find that the summary provided in the abstract is useful all by itself. (Beware: sometimes authors may exaggerate or oversimplify information in the abstract. More about that in the next section.)
How to read a research article
You have found and selected an article to read, and now it’s time to buckle down and read. This is not as easy as it sounds. For many of us, education research looks like a foreign language. However, like any language, this genre also has its own syntax, grammar, and idioms which can provide a rich conversation…when properly understood.
It helps to understand the structure of the typical research paper first. Although the specifics vary from journal to journal, most research articles have the following basic parts:
Armed with this knowledge of the parts of a research article, here is the workflow that I use regularly for processing and evaluating a research paper. It helps me get through the reading fairly quickly, but still allows me absorb the information that I need to learn. Remember this is the main reason we read research articles: to learn something that we can put into action.
- Read the introduction with the goal of understanding the research questions. Ask yourself as you read the introduction: Why is this study being done? What problem is the study trying to solve, or what questions is it trying to answer? Sometimes it helps to start at the end of the introduction first, because that’s where the research questions are often actually posed, and then loop back to the beginning and start over. I also have a rule that if the research questions are not made explicit in the beginning of the paper, I stop reading the paper. A research study that cannot clearly state its own research questions is likely to be a waste of time!
- Read the literature review with the goal of understanding the big picture … don’t necessarily chase every reference. The literature review is in many ways the most important part of the paper. It sets up the context for the study, introduces vocabulary that will be used, and provides more research that you can read later. However, the lengthy list of quotations and references might seem overwhelming. Don’t sweat the details of the references. Instead, look for the big picture: Given the research questions of this study, what seems to be known already? what nomenclature is used to describe the concepts? where does this study fit into the overall tapestry of research already done? A well-written paper will make all of this clear in simple language, while demonstrating due diligence in checking what has already been published. A paper that cannot explain its own context, or which fails to find any prior related research as a point of reference, again is probably going to be a waste of time. You may stop reading it now.
- Read the methods section to understand the context of the study and start considering its limitations. Questions to ask when reading the methods section include: Who is being studied, and where? How are the questions being addressed? What are the overall logistics of how the subjects in the study were studied? What kind of surveys or other instruments were used? Here, it is important to start thinking critically about the study. Suppose the study is asking whether students in a flipped section of introductory biology perform better on the final exam than students in a non-flipped section. All kinds of question arise: How many students were studied? Were the flipped and non-flipped sections comparable in terms of size, academic level, gender, time of day, and so on? Were the two sections taught by the same person? Was the final exam written by that person? Was it graded only by that person or was there someone else? A good methods section will address the validity of the study in many ways and it should provide enough information that any inspired reader could replicate that study at their own university or school.
- Skim the results section but do not read it in depth. This sounds lazy and counterproductive. Something called the “results section” sounds like the one part of the paper we must read! But the fact is that much of the statistical esoterica that show up in the results section are generally not for the reader — they are mostly for the peer reviewers. The rest of us should generally understand what the results were, and whether any quantitative results are statistically significant. But the details of the ANOVAs, t-tests, and so on are not intended to convince the reader that the conclusions (next section) are right; they exist to convince the journals that this study employed good methods. Many lay readers of research get bogged down in the results section and give up, because they don’t understand or are bored by the statistics. I discovered that I can still get actionable information out of a paper even if I skim (or even completely skip) the results section, if I carefully read the other parts.
- Read the discussion section with the goal of seeing what the authors think the results say. The discussion section is where the authors unpack the results, so this is worth reading in depth. As with the methods section, the discussion section is a place to put on our critical thinking caps. Ask yourself: Are the authors authentically and scientifically interpreting what their data say? Instead, are they evangelizing for their own pet theory? Or spinning the results to give a favorable impression of their hypotheses? You may have a very different interpretation of what the authors’ data say than the authors themselves, having seen the context and methods of the section. In the example of the flipped vs. non-flipped sections of the biology class, the authors might have found a statistically significant difference in the final exam scores of the two sections with the flipped section performing better. That’s good, but more care is needed: Is the result likely to be applicable to courses outside biology, or courses beyond the intro level? Were there any confounding variables introduced during data collection that might have influenced the results? Keep asking probing questions about what the authors are saying.
- Read the conclusion section with the goal of gaining more context. The conclusions section is where many authors will provide insights to the limitations of the study that we didn’t think of ourselves. This is a very good sign because it shows the authors are willing to be proven wrong. It is also a place where future research questions are posed, so potentially you could pick up where the authors left off.
As a final step, I often loop back around to the abstract and re-read it, having read the whole paper, to see if what the authors say the study found is the same as what the study actually found. Looking at the abstract with fresh eyes can often reveal a bias by the authors toward their own hypotheses, which should drive you to think even more critically about the data and the results.
Acting on what you learn
As I stated in the previous section, the whole purpose of doing and reading research is to learn things you can put into action, especially in your own teaching. Once you have finished reading an article, take a few minutes to think about how what you read could be useful in your own teaching on a practical level. How will you use the results to inform your own practice? Or, could you possibly replicate the study in your own setting and add to the pool of knowledge? One of the great things about flipped learning research is that it is largely done by ordinary instructors in their own classes to understand how best to help their students, so it’s wide open for people to join in.
Editor’s Note: In future issues of this newsletter, Dr. Talbert will apply his own workflow method to introduce and analyze 1 or 2 recent flipped learning studies for our consideration right here in the Flipping Research Corner.