Evidence-Based Practice Tools for Practicing Clinicians

Welcome to Evidence-Based Practice Tools for Practicing Clinicians, brought to you by ASHA’s National Center for Evidence-Based Practice in Communication Disorders, also known as N-CEP In this course, we will talk about a broad overview of evidence-based practice and the ASHA resources that can be used to guide and support your clinical decision making At ASHA, we strive to transform the professions and the Association by providing clinicians with the resources to help them demonstrate their value and improve the quality of their services Here at N-CEP, our goal is to educate our members about evidence-based practice and give them the necessary tools to make evidence-based decisions I am Mariel Solomon, the presenter of today’s presentation I am a part of ASHA’s National Center for Evidence-Based Practice and therefore receive a salary from ASHA As a non-financial disclosure, I am an ASHA certified SLP and ASHA member The learning objectives of today’s presentation are to… define and identify components of evidence-based practice, describe the four steps of evidence-based practice, define and create a PICO question, identify factors that may influence study quality, and use ASHA resources to enhance evidence-based practice Here is how we will achieve those learning outcomes… We will walk through a review of evidence-based practice, which will also be referenced as EBP in this presentation, and review its principles We will take a closer look at each step of the EBP process, identify various resources to aid in engagement, and finally, we will provide some clinical examples to show you how to integrate EBP into your clinical decision-making Let’s start off by talking about what we mean when we say evidence-based practice EBP is a frequently used term in clinical communities Each profession has its own variation of the definition In the SLP and Audiology professions, evidence-based practice is a framework integrating external scientific evidence, clinical expertise, and client perspective to answer clinical questions and make informed decisions ASHA’s position statement views the principles of evidence-based practice as necessary to provide high quality, individualized clinical care Let’s take a closer look at each pillar of EBP One component of EBP is external scientific evidence, which refers to research outside of everyday clinical practice, usually from methodologically sound research External scientific evidence comes from individual studies, synthesized evidence in the form of systematic reviews and meta-analyses, and also evidence-based clinical practice guidelines First we’ll talk about individual studies They attempt to answer a specific research question There are many types of individual studies and the specific design depends on the study’s purpose In experimental and quasi-experimental studies, the investigator manipulates one or more variables in order to compare those that received the manipulated condition to those that did not An example is the comparison of children receiving a language treatment versus children assigned to a wait list control group In experimental studies, the participants are randomized to the experimental or controlled conditions and in quasi-experimental studies, they are not randomized In observational studies, there is no experimental manipulation Let’s look at some more examples Here is an example of an experimental study Researchers are investigating the effect of a new hearing aid to improve auditory function They randomly assign participants into three groups: those receiving the hearing aid under investigation, those receiving a different hearing aid, or those without any hearing aid intervention The researchers’ random assignment of participants and the use of comparison groups makes this study experimental If these participants were not randomized into these groups, this would be considered a quasi-experimental design If a study investigated the outcomes of a group of adults with hearing loss who all use the same hearing aids, it would be considered an observational study because there was no manipulation of the variables and no comparison group Another form of external scientific evidence comes from synthesized research Synthesized evidence combines the findings from individual studies and provides conclusions about a body of evidence

Synthesized evidence that use systematic methods are called systematic reviews and meta-analyses According to the Cochrane Library, a systematic review identifies, appraises, and synthesizes empirical evidence (typically individual studies) that meet predetermined eligibility, and quality criteria to answer a clinical or research question Those conducting the systematic review then provide qualitative conclusions based on the included studies Some systematic reviews also include meta-analyses A meta-analysis is conducted in the same manner as a systematic review The identifiable difference is that a meta-analysis uses statistical methods to summarize the results of the studies into a pooled quantitative effect size and an associated confidence interval A meta-analysis helps to determine the overall treatment effect or effect size across studies and if these treatment effects are consistent from one study to the next These additional statistical measures can give clinicians a better picture of the clinical significance of an investigated relationship Both synthesized evidence and individual studies have advantages and disadvantages As synthesized evidence includes a number of studies, there is an opportunity to compare the various outcomes of the studies that use different methodologies to determine if there are similar conclusions Systematic reviews also use a predetermined process and typically assess each included study for its quality and level of evidence This transparency is intended to limit bias and offers greater objectivity Therefore, clinicians may have more confidence in the findings from a systematic review because its conclusions are drawn from a body of evidence and not just a single study On the other hand, there could be more individual studies that have a very specific focus because, unlike systematic reviews, individual studies are not dependent on the need to have an already existing corpus of evidence There are, however, some disadvantages to using an individual study alone The first disadvantage is the sample size of a study If an individual study has a small number of participants it is not representative of the population, so outcomes of the study may not be generalizable to the larger population Methodology and bias can also be disadvantages Studies that are poorly designed and developed without methodological rigor are susceptible to introducing bias into the results and limiting the conclusions that can be drawn A final source of external scientific evidence are evidence-based clinical practice guidelines These guidelines are documents that provide recommendations to optimize delivery of SLP or audiology services within a specific clinical topic or population They are typically developed by a group of subject matter experts and are informed by a systematic review of the evidence Evidence-based guidelines use the strength, direction, and magnitude of the evidence to formulate recommendations for clinical practice Other guidelines provide recommendations on a summary of opinions of subject-matter experts and these are considered clinical expertise, the next component of the EBP triangle As stated by Dollaghan, clinical expertise can be best described as the evidence that is internal to our clinical practice It refers to our own training, knowledge, and expertise of speech, language, swallowing, and hearing development and disorders It also refers to the expertise of our colleagues The Practice Portal, found on ASHA’s website, is considered clinical expertise because they are developed with subject matter experts Consensus-based documents, such as ASHA’s Preferred Practice Policies, are also considered clinical expertise Clinical expertise can also take other forms When a clinical practice guideline is created without a systematic review or when the evidence is insufficient, a guideline group may formulate consensus recommendations for clinical practice based on expert knowledge and experience The final component to complete the EBP triangle is the perspective of the client, family member, or caregiver These perspectives are important because they can provide insight into how clients and caregivers feel about the services they are receiving This is also known as social validity Perspectives can come directly from the client or can be found in either quantitative or qualitative studies, as well as systematic reviews

Opinions from the individuals who receive audiology or speech-language pathology services are very important to clinicians Client perspectives are necessary to judge if the intervention is important to the client’s preferences and culture As clinicians, we want to ensure that we are addressing outcomes that are important or of interest to each individual client For example: Even though an intervention is supported with high quality external evidence and clinical expertise, can its long-term effect be seen if the client or caregiver does not like it, does not actively participate in it, or finds it too difficult to implement? Each part of the EBP triangle is important and each component of EBP, external scientific evidence, clinical expertise, and client perspectives, needs to be balanced and weighed in order to make the best clinical decision for your client Now that we have reviewed the components of EBP, let’s talk about the steps involved to integrate EBP into your clinical decision making The four steps are… framing the clinical question, finding the evidence, assessing the evidence, and finally, using the evidence to make your clinical decision The first step in applying evidence to a clinical decision is to frame the specific question for which you want evidence One widely used approach to help you frame your question is known as PICO, which stands for population, intervention, comparison, and outcome The PICO format is used to help you narrow down the specifics of your clinical question “P” is the population of your clinical question It can be a very specific population such as military veterans with acquired brain injury or a broad population such as individuals with speech and language impairment “I” is the intervention or assessment used in the clinical situation such as diet modifications, hearing devices, or receptive-expressive language assessment tools “C” is the comparison, which is what you’re comparing your targeted intervention to This can be no treatment, a sham or fake treatment, or treatment-as-usual “O” is the outcome of the interventions received, such as vocabulary ability, receptive language skills, or safe swallowing Here is an example of a PICO question In individuals with dementia what is the effect of spaced-retrieval memory training compared to no treatment on memory skills in activities of daily living? Let’s breakdown the question: The population of the question is individuals with dementia The intervention is spaced-retrieval memory training The comparison is no treatment And the outcomes are memory skills in activities of daily living Once a clinical question is formed, components of the PICO question can be used as keywords to search for relevant evidence and keep the search relevant to our objectives For example, an audiologist wants to know the most effective treatment for individuals with benign paroxysmal positional vertigo or BPPV From that question, the clinician can use the PICO elements of “lateral canal BPPV”, “interventions”, “maneuvers,” and “vertigo” as keywords in their search It is important to note that not all clinical questions will have every aspect of the PICO format, but each aspect will help increase the specificity of the results In this example, the clinical question only includes population, intervention, and outcome Once you have your question and key words, there are a number of places you can go to search for the evidence such as the Cochrane Library, Education Resources Information Center (which is also called ERIC) There is PubMed, a free resource from the National Centre for Biotechnology Information at the U.S. National Library of Medicine And there is ASHA’s own ASHAWire, which is an online library for all of ASHA’s publications ASHA’s publications include these four peer-reviewed journals, which ASHA members can access through their accounts Now that you have asked a clinical question and found information to answer your question, the next step is to assess the quality of the evidence This is a crucial step in order to identify possible methodological bias that could influence the results There are a number of tools available to assess the methodological rigor of individual studies

Oftentimes these tools may ask specific questions based on the study design For example, tools that assess the quality of randomized controlled trials, may ask questions like, “were subjects randomly assigned to the intervention and control group?” and, “Were the assessors and clinicians blinded to the treatment condition?” Other tools that evaluate the quality of single subject design studies ask questions like, “was there sufficient sampling during the baseline and treatment phase” and, “Was there selective reporting of outcomes?” And finally, tools used to rate the quality of observational studies include questions such as, “Did the participants adhere to the intervention protocol?” Just like individual studies, synthesized evidence from systematic reviews and meta-analyses can also be of varying quality and should be appraised There are a number of available appraisal tools to assess the quality of synthesized evidence: there is AMSTAR, or the Critical Appraisal tools from the Oxford Centre for Evidence-Based Medicine The National Institutes of Health and the CASP also have resources to help determine the quality of synthesized evidence As we can see here, these tools evaluate similar things in a systematic review or meta-analysis such as specific mention of a clinical or PICO question, clearly stated inclusion and exclusion criteria of studies, and the quality assessment of individual studies And finally, guidelines should also be assessed A prominent and commonly used appraisal tool for clinical practice guidelines is the Appraisal Guidelines for Research and Evaluation II also known as AGREE It consists of 23 criteria across the six domains that you see here The six domains were designed to assess the applicability, comprehensiveness, and methodological rigor of guidelines Once you have found the evidence and have assessed it, you can use this information along with your clinical expertise and the perspectives of the client and family to guide you in a clinical decision While this might seem straightforward, making the clinical decision can be complicated At times, there can be a lack of evidence aligned with all aspects of your clinical question When this occurs, clinicians should use their expertise and knowledge of the client to determine if the available evidence could be extrapolated and applied to the individual receiving services Limited research on topics is only one barrier There are a number of other real barriers that can also prevent clinicians from fully and appropriately engaging in EBP These may include the ability to access relevant research, the lack of quality evidence, and most of all, the lack of time Clinicians have limited time on their hands to systematically search for, read, and analyze, information Because of these barriers, ASHA has developed resources and tools to help clinicians engage in EBP The Practice Portal is an ASHA-based resource on a variety of clinical topics and professional issues Clinical topics are organized for the clinician to assist with the delivery of care Individual portals of clinical topics typically have a general overview of the population or condition, incidence and prevalence, signs and symptoms, causes, roles and responsibilities of the clinician, assessment, and treatment Each Practice Portal is developed with content vetted by subject matter experts The Practice Portal provides clinical expertise in patient care, which as you remember is only one part of the EBP triangle This tool is not designed to dictate one’s practice, but is designed to be an additional guide, in conjunction with other resources, for the clinician in his or her decision-making Another ASHA resource is the Evidence Maps, which are curated by N-CEP The ASHA Evidence Maps are a searchable online tool used to save clinicians time and to guide evidence-based decision-making Articles cover a range of clinical SLP and audiology topics across the lifespan The Evidence Maps only include articles that are systematic reviews, meta- analyses, and clinical practice guidelines to emphasize external scientific evidence, clinical expertise, and client perspectives Let’s use the Evidence Maps to help answer a clinical question Remember the PICO question example about dementia that was used earlier? Let’s expand it to include not just spaced-retrieval training but to include any cognitive intervention From the homepage, you see an Evidence Map on Dementia Once you select that map, you can see a list of brief article summaries that are categorized by the three components of EBP

Articles can be refined using various dynamic filters and publication date range can be adjusted Looking closer at one of those articles on the dementia map page, we can see the article type, citation information, a link to the full summary for more details, and a link to the article Article links will take you to the website or database to access the full-text of the article Some articles may be open access, which means that they are free, but some articles may require a fee from the publisher In the brief article summaries, conclusions and recommendations are labeled with the appropriate EBP component Summarized conclusions and recommendations allow the clinician to quickly determine relevance of the article and synthesize information to their PICO question Now let’s go into the full summary of that clinical practice guideline Again, you see the citation information In the full summary you can see the article’s quality rating, and the evidence ratings for the guideline if they were provided in the document Clicking into the full summary of a systematic review, we see a similar format of citation information, quality indicators, and full descriptions of clinical/research questions addressed, population, investigated intervention or assessment, and number of included studies ASHA’s Evidence Maps tool saves clinicians a step in the EBP process by providing quality indicators for the various articles included in the maps Systematic reviews are evaluated across a number of criteria such as a search strategy that is described in sufficient detail for replication and reproducible quality assessments Guidelines are also evaluated using a modified version of the Appraisal Guidelines for Research and Evaluation II instrument If a document addresses multiple conditions or populations, it may be included in multiple evidence map The recommendations and conclusions for that specific condition or population can be found under that heading The ‘Go to Map’ link will send you to that map for more about that topic So now that we have gone over the steps of EBP and talked about some of the ASHA resources available to help overcome some of the barriers to engaging in EBP, let’s take a look at a few case examples and highlight how these ASHA resources can be integrated into the EBP process We’ll start with this scenario: You are a school-based SLP with a new caseload of early elementary students You are used to working with older, middle school-age children You have not worked with younger students since your clinical fellowship, so you want more information about younger students with learning disabilities and language disorders Being up to date with available evidence would help you evaluate and treat other students on your caseload You notice that Marcus, a five-year old boy in kindergarten, has a multidisciplinary evaluation due soon He entered the elementary school with only receiving SLP services for a language disorder, but has been referred for a multidisciplinary evaluation due to the concerns of a possible learning disability You want current information about assessing young students with spoken language disorders and learning disabilities From this situation, you have enough information to create a clinical question, which is the first step of the EBP process As stated previously, clinical questions do not always have to follow the PICO question format A useful clinical question, however, will typically contain at least some elements from the PICO format So our clinical question becomes, “What is the best way to assess a five year-old student with a language disorder and a suspected learning disability?” Now that you have a clinical question, the next step is to go out and look for evidence You can start by going to ASHA’s Practice Portal to get a better understanding of the disorders that you are dealing with You go to the Spoken Language Disorders practice portal It has a lot of useful information for the oral language portion of Marcus’ evaluation, but this practice portal page does not address reading and writing deficits You decide to find other Portal pages and click on the Written Language Disorders practice portal That is where you find more comprehensive information about reading and writing deficits, which is extremely relevant to students with learning disabilities The ‘Assessment’ section provides important considerations and key components for Marcus’ evaluation The Practice Portal also provides links to the Written Language Disorders Evidence Map where you can find research about these key components

That link takes you directly to the assessment section of the evidence map The list is already filtered for articles about assessment and the list is a very manageable number You quickly scan the article summaries that are already separated by the EBP component You find a systematic review and click on the link to view the full summary This full summary provides the quality of the article, which saves you a step in the EBP process It also provides more details of the review and the conclusions to help you determine if the article applies to Marcus’ situation It offers a link to receive the full text or a link to where the full text can be purchased From visiting ASHA resources, you already have two of the three components of EBP: For external scientific evidence, you have a systematic review of correlated factors that can predict later literacy outcomes You can also use the Evidence Maps to find the latest research about those literacy outcomes You can incorporate this evidence with the clinical expertise from the ASHA Practice Portal, and your own clinical experiences, to guide the diagnostic approach and to provide information to the multidisciplinary team in order to determine the most appropriate services for Marcus In order to complete the EBP triangle with client perspectives, we cannot forget to consider if these diagnostic approaches are appropriate for Marcus’ needs and the concerns of his family In this second scenario, you are a speech-language pathologist at a growing hospital The administration is initiating development of standard protocols for individuals admitted to your hospital with stroke Providing standard expectations for management of particular groups is sometimes called a ‘clinical pathway’ As the SLP, you are required to help develop the protocols for provision of speech-language pathology services based on evidence from existing clinical practice guidelines You need to identify best practices in the management of stroke patients and start by looking for stroke clinical practice guidelines But, how do you find the right guideline? Your PICO question may begin very broadly, such as: Which speech pathology screens, assessments, and interventions lead to optimal communications and swallowing outcomes in adults after stroke? Knowing that the evidence maps provide up-to-date summaries of the latest clinical research, you decide to start here to save time On the homepage, you quickly search for the keywords: “stroke clinical practice guideline.” This produces three pages of results that appear relevant You can see they are presented in a list, with a title and short description of each article You click on the first result, a guideline from 2007 This guideline is Highly Recommended ASHA has helped you assess the quality of the evidence, saving you time in the four steps of EBP You can read about ASHA’s guideline rating process here In this guideline, you see all the recommendations laid out for SLP services, including dysphagia assessment, aphasia treatment, and more These recommendations have grades corresponding to the strength of the recommendation You can find a key to these grades under ‘Evidence Ratings for this Document’ While the article was relevant, you were only able to see title and description of articles from the search page You would like to see the year of publication for each guideline to ensure recommendations are current You also want to refine your results by assessment or treatment You can do that by going directly into the Stroke Map That map can be found on the Evidence Maps homepage Once inside the stroke map, you need to find clinical practice guidelines You can use the orange boxes to find articles of a specific type, in your case, guidelines Articles are listed by publication date, which is found in the citation You find a guideline from the American Heart Association and American Stroke Association from 2016 Here are numerous applicable recommendations You can filter results as you investigate different aspects of speech-language pathology services in stroke You may want to focus on assessment only, or only see results on aphasia For example, when deciding how to address dysphagia screening of individuals with stroke at your hospital, you can filter by Screening and Dysphagia Once you click to enter specific guidelines, you are able to print out full text from this

guideline for further review You can use the document to provide back-up for your recommendations during discussions with administration After reviewing results from various guidelines, you are able to synthesize the recommendations for best practices Considering this evidence and your hospital’s unique needs, you can consult during development of a clinical pathway to standardize the provision of swallowing and communication services after stroke You are able to advocate with evidence for inclusion of speech-language pathology services in this clinical pathway at your facility Here is our final scenario You’re an early intervention Audiologist You hear about upcoming legislation that, if passed, would result in funding cuts for local early intervention programs As a member of your state association, you want to find research about the impact of audiology in early intervention services You decide to search for evidence about the outcomes of early intervention services to show the value of audiology and advocate for legislative policies that support best practices Before you begin reviewing the literature, you develop a PICO question to guide your search Your focus is on the population of infants and toddlers The intervention is early hearing detection and intervention programs You want to compare young children who receive early intervention services and those who do not have access to those supports The outcomes should tell you about the child’s development in the areas of hearing and communication Together, your PICO question becomes: What are the effects of early intervention services for hearing impairments on communication and education outcomes of children with hearing impairments? Before you begin your search, you may first try to use a resource like Google Scholar or some other database With this type of search tool, you may find nearly half a million results Most likely, you don’t have time to sort through and review that many articles Instead of feeling overwhelmed, you decide to go to ASHA’s Practice Portal to better inform your question At ASHA’s Practice Portal, you find information on best practices for newborn hearing screening This clinical expertise builds one piece for your EBP triangle But, you decide you need more specific research about early intervention services to present legislators So you then choose to go to ASHA’s Evidence Maps You enter the newborn hearing loss map and choose to filter your search After reviewing the filter options, you select “Service Delivery” and “Early Intervention” By selecting one of the tabs at the top, you can choose evidence to build each side of your EBP triangle With the “External Scientific Evidence” tab, you get systematic reviews about the outcomes and effectiveness of Audiology services Later, you can review the other tabs to find guidelines with clinical expertise and systematic reviews with client perspectives You now have a manageable body of evidence to review As you review each article, you find that each one has already been appraised for quality by clinical staff at ASHA You can determine whether the evidence provided in the article meets an appropriate level of quality for your needs Finally, after reviewing all of the conclusions and recommendations, you have gathered a body of evidence showing that early intervention for hearing impairments does result in positive outcomes for later communication and educational development You are now armed with evidence and feel ready to advocate to your local legislators for the laws that will result in the best outcomes for young children So here is what we learned today EBP is a balance of external scientific evidence, clinical expertise, and client perspective The EBP process consists of: framing the clinical question, which can be formed as a PICO question Remember, PICO questions contain population, intervention, comparison, and outcome Once we have a clinical question, we then find the evidence, assess the evidence, and then make the clinical decision There are multiple appraisal tools for individual studies, systematic reviews, and clinical practice guidelines The EBP process can be a cumbersome one and, as demonstrated in the clinical scenarios, ASHA resources such as ASHAWire, the Practice Portal, and the Evidence

Maps, can be used to streamline the process to make a clinical decision It is important to remember that, although ASHA resources and other resources may be available to you to ease the burden, it is up to you, the clinician, to incorporate the EBP components to make the appropriate clinical decision For more information about evidence-based practice and its process, go to asha.org/research/ebp ASHA’s EBP page also offers links to other resources and practice tutorials for every step of your clinical decision-making process Thank you for learning more about evidence-based practice in communication disorders and we hope this will inform you in your clinical practice If you have any questions, please check out ASHA’s EBP webpages or email us at [email protected]