Yesterday we talked about how to find research sources. Today we’re going to discuss evaluating the strength and value of the research you’ve found.
First consider the title and abstract/summary. Don’t download and try to read everything that’s on the topic. For instance you may be looking for research to write an English Literature essay on George R R Martin’s A Song of Ice and Fire and find a lot of information about the historical inspirations for the series. Unless you were looking specifically for historical influences, that research is not likely to help you. Yes, you’ve found a critical article or book on your topic, but it’s not a piece of research that is likely to help you meet your objectives.
Whether you’re using a database or google, a full-length book or an article, you should look up information about the author and their credentials. Usually this should be supplied for you in the form of an author bionote. Why is this information important? Well first, you want to see what’s their academic background. As a fantasy literature scholar I tend to come across articles written from all areas and specialities – history, sociology, psychology, education, etc. Even if the article is written by a literature scholar, that doesn’t guarantee that contemporary fantasy is their area of speciality. Their primary area of focus may be something like medieval literature or children’s literature. By understanding their background, you can grasp some understanding of their research foundation. For example, a medieval literature scholar, might have a good grasp of historical knowledge (both in terms of actual facts as well as critical theories and methodologies in that field) but may be unaware of current literary criticism on fantasy. By understanding their foundation, you can evaluate the application of their work to your own research along with possible strengths and weaknesses of their work. Also consider the journal itself or, if it’s a book, the publisher and whether the book is part of a series. What’s their remit? I.e what is their area and speciality?
If you’re using the internet or you’ve picked up a book at a local bookstore instead of a university library, take a moment to consider the intended audience. (This goes for EVERYTHING you read and not just when you’re in research mode. Think critically.) Be aware that the website or book is aimed at the general public and not at an academic audience. That isn’t to say that the information published in these works are incorrect, but approach them with the understanding that their methodology or underlying assumptions may not be thoroughly vetted and backed by published peer-reviewed research. For example, there are a lot of parenting books out there written by popular bloggers. They may have done some “research” themselves in the form of reading other parenting books or maybe even peer-reviewed articles. But they are unlikely to have the sort of training where they can critically examine this research and evaluate their methodologies and assumptions. So look up author credentials and see if they’re speaking as a reputable researcher in the field. Don’t assume that a published book means the author and their work was thoroughly vetted by the publisher. The author might be someone who can guarantee to the publisher that their book will sell because they already have a wide audience; They could be a public figure or even just someone who can show some decent social media following.
If the author IS esteemed then the next step is to evaluate the appropriateness of the work in your field. Popular non-fiction is a tricky case. For instance, while I used the scientific theories in Stephen Hawking’s A Brief History of Time as a critical tool to investigate Fantasy literature in my first monograph, it’s probably not a book that a theoretical physicist should be basing their work on; you should find the original peer-reviewed research as it’s applicable to your field.
Once you’ve thoroughly vetted the academic rigour of the author and publisher, the next step is to take a good look at the date of publication. If it’s an older print, then new evidence might have been published since then that refutes or challenges the article. In some cases, the author themselves might have built on and expanded the work (or retracted it!) and you should certainly be aware of these as well.
Finally, after vetting the publisher, the author, and the date of publication, we’ve reached the part where we’re sitting down to read the article or book itself. But wait! This doesn’t mean we can turn off our critical lens. You should now evaluate the author’s methodology. Are the steps they’ve taken to reach their conclusions appropriate? Are there any gaps that they fail to acknowledge? If they have acknowledged these gaps, is their justification for why they’ve left the gap in place reasonable? I.e did they properly consider these gaps and the choices they made for their methodology, or did they put it in as an afterthought once the research was already undertaken (in these cases, they’re probably responding to peer review feedback). You can still go ahead and utilise the piece in your own research. But acknowledge their gaps.
Vetting your sources doesn’t necessarily mean you’re don’t use them at all. Instead, it means you should carefully consider their weaknesses. This consideration should be part of your research notes. I.e not necessarily part of your research paper itself. But if the piece of research becomes an essential foundation to your own work, then your paper should absolutely identify these weaknesses. Make sure to keep the tone professional (i.e. no personal attacks or harsh judgment). You should also include how you’re avoiding the same pitfalls in your own work or offer your own justifications. A note of caution here: don’t go overboard with spending so much time (word count) defending your methodology that you don’t have enough space for the rest of the paper. If you’re using an outline, roughly identify how big each section should be before you start your first draft. You might spend hours or days critically examining these sources only for it to end up as 1 or 2 sentences in your work. I’m sorry to inform you but that’s what the research process looks like. 🤷🏽♀️
Next time we’ll talk more about the research process and how to identify a strong argument. Until then, take care!
Doc Fantasy Reads 