What Is Systematic Literature Review Information Technology Essay

What Is organized Literature Review In make foration Technology Essay existential softw be engineering requires the scientific employment of qualitative and quantitative entropy to realize and improve the softw be product and serve well. To conduct empirical studies, the treat counsel steps to be performed and investigate system ar really essential elements 11. Apart from the traditional regularity much(prenominal) as experiments, case studies and surveys, the organized belles-lettres appraise, withal called taxonomic recap, is another strategy becoming normally used in empirical softwargon engineering in order to differentiate pertinent empirical studies and combine leave alones to provide reliable information for interrogationer.This paper aims to provide the general idea of self-opinionated critique, wreak to conduct the suss out according to the vivacious road map and suggestion to improve the guideline. The paper is structured as follows section 1 giv es an introduction to imperious check over, explains the reason why opinionated polish up is regarded, discusses the differences from history survey and pros and cons of taxonomical review. Section 2 explains growth of conducting arrogant review in detail according to Kitchenhams guideline 1. Since in that location be dummy up close to re inflexibleions and gaps could be im prove in the guideline, section 3 set forth those limitations and provides suggestions for improvement. Lastly, section 4 presents some concluding remarks.1.1 What is Systematic literature review?Systematic literature review is the methodology that lookers use to gather and evaluate available information to specific re await root word. The use of a SLR is mainly proposed to provide an indifferent(p) and opinionated approach to answer a structural unbelief which focuses to explore topics. The studies or articles that used for organized review are called primary election studies, the systemat ic review itself is considered as the secondary studies. The review process is very formal with strict procedures and sequence. Each step of the process must be well- delimitate and lot be reproducible by other re calculateers. The holdion criterion of primary studies and type of acceptable information are alike defined before hand and exhaustively reviews. To identify the primary studies is done by the use of take careing technique. All search methodology and cream criteria are transparent for the reader by described in review communications communications communications protocol. Thus, this means other readers can replicate this review in systematic appearance. Finally, selected primary studies are analyzed and aggregated, in which forming a systematic review for answer the research wonders.1.2 Why Systematic reviews is take a aired?Originally, systematic review has been introduced in medical and clinical field as a way to help clinical practitioner finds the answer to their question relating to their practices. Before systematic review was implemented, researchers gull ticklishies in several areas that traditional review cannot satisfy their need. in that respect is no structural way to review primary studies and to en real that all related evidence has been evaluated. To summarize and evaluate the knowledge from primary studies is very severe and also because each pack may use different design and organization of information may be vary. The result from traditional review is difficult to evaluation if there is conflicting result. Many fourth dimensions, reader of the review let some doubt to the quality of researchers work since the review method is not outdoors and denotive. In addition, due to broad scope of traditional review, the result of the review can easily be bias as the selection choice by reviewer may not be consistency without well defined guideline of the selection. Thus, systematic review methodology has been introduced to address those difficulties.Systematic review process have increasingly recognized and replaced traditional reviews in many academic fields including software engineering discipline since it provides effective way to summarize and jibe research result to help those researchers in their studies. The researchers can utilize systematic review to design new studies that previously difficult in the past with traditional review. With systematic review, it helps reducing reviewer bias since it uses objective and reproducible criteria for primary sources selection with strict assessment of the resources. It also help s researchers to combine result from several small studies in which can help them conclude more precise and dependable result. Moreover, it assists in identifying gap that researchers could make further investigation 1.1.3 Differences amongst Systematic and Unsystematic reviewThere are several key differences between systematic review and unsystematic review or traditional rev iew. This topic entrust discuss each difference of these two types of review.First, the search for primary studies of these two reviews is widely different, with traditional review, the scope of primary studies search usually have wide range of coverage with no strict rules how to search. In contrast, the search of primary studies in systematic review is very focused on the topics, researchers need to identify question and predefined search rules that can be reproducible by other researchers.Second, the primary sources selection process is also different. With systematic review, selection process has to be prede terminalined researchers must decide which type of resources is acceptable for the review so that the selection process is explicit and transparent, and then select primary sources with these strict rules and criteria. Unlike systematic review, traditional review selection process does not have specified criteria on the selection process and dependent on researchers experie nces to select the primary sources.Finally, the evaluation method of systematic review followed strict evaluation rules to evaluate each of selected primary sources while traditional review is variable establish on individual researchers methods.1.4 Advantages and disadvantages of Systematic reviewThough there are several advantages that researchers can utilize systematic review method in their studies, there are still some disadvantages remain comparing with traditional review. Researchers depart need to aware and select review type that is appropriate with their situation. This topic bequeath discuss both advantage and disadvantage of systematic review.Advantages One of the major advantages of the systematic review is that it improves the precision and completeness of the result. Because systematic review process uses well-defined method to search and select for primary studies in which allow for result in less biased sources comparing with traditional review The systematic rev iew process helps researchers to identify the consistency or inconsistency of the result from its selection process. If result is consistent, it provides strong answer to researchers questions. If not, researchers can identify the gap and then study the variance. This earn will be difficult to identify through traditional review process. Systematic review can apply statistic technique (meta-analysis) to help combining data from more than one primary studies which will give more precise answer to researchers question than using whole one primary study. With traditional review, it will be more difficult for researchers to compare and conclude the result from several primary studies to answer their focus question.Disadvantages Because of its limited focus and predefined method, as it is the advantage of systematic review, it can be disadvantage in some cases. Since the process does not allow comprehensive coverage of the evidence to answer research topics. Thus, researchers must car efully determine their condition and select technique appropriately. In general, traditional review is more useful if researchers want to obtain more broad perspective of their research topic comparing to systematic review. Traditional review is more useful if the researched topic is the brand new topic, since not much primary studies will be available. Traditional review will have more information coverage than systematic review. Due to its strictness in process and methodology, systematic review process usually needs more time and effort from researchers to conduct the review.2. Systematic review guidelinesAs prime(prenominal) introduced in medical research studied, systematic review has also been brought to software engineering field by B. Kitchenham (2004). Since software engineering research method is less strict and less experimented-dependent comparing with medical studies, the modification is needed in the process of systematic review to adapt to the characteristic of soft ware engineering studies. The guidelines emphasize the distinction to medical systematic reviews and guide the software engineering researchers how to perform a systematic review. There are several activities involves in the systematic literature review and they are specified in the guideline. In this section, I refer to the guideline suggested by Kitchenham 1 which describes a systematic review process into one-third main casts Planning the review, Conducting the review and Document or report or the review. Each of them consists of order of stages. The implementation each phase involves iteration, feedback and refinement in order to move to next stage and finally reach copacetic outcomes as lucubrated in figure 1. Please note that there are some more optional stages described in the guideline. But I only refer to the stages that are essential to be performedFig. 1. Phases in Systematic Review2.1 Planning the reviewIn the first phase of the review, the final outcome which would be produced is a review protocol. It is considered as a propose which defines the research questions that will be addressed by the review and basic review procedures. The planning phase consists of the following stages Identification of the need for a review Specifying the research question(s) Developing a review protocol Evaluating the review protocol2.1.1 Identification of the need for a reviewBefore conducting the review, the reasons behind why the systematic review could answer the research questions or could be useful for further must be given. Originally, the need for a systematic review initiates from the demand to fairly sum up all existing information about some phenomenon. Probably the reasons are to translate more general conclusions instead of just obtaining from individual studies, or may be carried out to lead up to further research activities. In particular, Kitchenham notes that the researchers should first make sure that a new systematic review is really needed b efore starting the review. And they should consider finding any existing systematic reviews relates to the topic of interest. It could be possible that they do not even need a new systematic review if there existed. Besides, the already published systematic review could help construct a protocol.2.1.2 Specifying the research question(s)This is the roughly important stage of the systematic review process. The research questions can be seen as a goal of the review since they drive the whole process of systematic review. To be in detail, the search process is conducted with the aim to classify primary studies that discuss the research questions. Furthermore, the data extraction and analysis processes must extract and synthesize the data in such a way to answer the questions.Kitchenham notes that asking the right question is the important issue in any systematic review. She provides some guideline questions to help in construct the correct questions. She also discusses the characterist ics and various types of research questions which are proper for the systematic review. For detailed structure of the questions, the PICOC (Population, Intervention, Comparison, Outcomes, and Context) criteria are used to define the review question elements. Besides, Kitchenham discusses about the different kinds of experimental designs to derive the acceptable studies. In particular, she argues whether studies based on primary studies of one particular type should be accepted for systematic reviews in software engineering.2.1.3 Developing a review protocolA review protocol is a concrete plan which details the process and strategy to perform a particular systematic review. A pre-defined protocol is crucial to minimize the possibility of researchers bias. The protocol contains all the essentials of the review and some other planning information. The elements described in protocol are background, research questions, aforethought(ip) search strategy, study selection criteria and proce dures, quality assessment criteria and procedures, data extraction strategy, data synthesis strategy and project timetable.Kitchenham suggests that the review protocol should be piloted during its development to find mistakes in the data search procedures, in which it can help to improve the review methodology.2.1.4 Evaluating the review protocolBecause the review protocol is significant for the systematic review, it should be evaluated before implementation. The evaluation procedures are done by asking researchers or experts to review the protocol and the agreements among all reviewers must be reached.2.2 conducting the reviewThis is the execution phase which follows the plan defined in the review protocol. The final outcomes of the systematic review are generated in the end of this phase. To conduct the review, the subsequent stages must be carried out Identification of research Selection of primary studies Study quality assessment Data extraction and monitoring Data synthesis2. 2.1 Identification of researchSince the systematic review aims to find all possible of available publications relating to the research question and make conclusion in a fair manner, how to generate the search strategy and publication bias are the critical issues that Kitchenham discusses in this stage. The search strategies defined in the protocol are used to discover the relevant publications. In general, the search strategies are done iteratively by trial searches using different combinations of search damage and in consultations with relevant experts. Typically the search term can be obtained by separate the research questions into individual elements base on PICOC criteria and then create a list of synonyms and relevant words. some other good way to derive the search term is by analyzing the heading of journals. Kitchenham notes that the search strategy should be designed to detect articles that report pessimistic results in order to illustrate researchers bias. other(a) major concerns regarding the systematic review are the completeness and repeatability. Kitchenham suggests that the review process must be transparent and replicable. By providing sufficient detail when documenting the review, this enables the study to be replicable and allows the external reader to evaluate the search terms. And the search terms should be put down properly.2.2.2 Selection of primary studiesThe purpose of the selection process is to assess if obtained primary studies have any actual relevance to the research questions. So that we can identify ones that provide direct information for the review. This process should proceed according to the plan defined in the protocol. Kitchenham explains that the study selection is a multistage process. Firstly, base on the research questions, the researchers should define the study criteria to point out the direct relevant studies. These inclusion body and exclusion criteria should be piloted to ensure the reliability and correctness when interpreted. The exclusion criteria should be applied first in order to exclude any irrelevant studies. Kitchenham suggests safekeeping the record of excluded publication with the reason of exclusion just after all unrelated publications have been filtered out. Then the inclusion criteria are applied to remaining studies. Kitchenham also mentions about how to increase trustworthiness of the process in an attempt to decrease the possibility of bias.2.2.3 Study quality assessmentAfter applying the inclusion and exclusion criteria to select the primary studies, the quality of primary studies is also considered significantly important to be assessed. Kitchenham describes how important of the quality assessment for slip, to allow researchers to evaluate differences in the study, and to weight the sizeableness of each study when synthesize the results. She also discusses about the hierarchy of the evidence described in medical guidelines. Base on their assumption, this hierarchy ca n be used to control the sorts of study included in the systematic review and it is a ground for the initial quality evaluation. To explain, the top of the hierarchy is the evidence from systematic reviews and controlled experiments, which is believed in the medical area that it is more reliable than the bottom level evidence such as the evidence from expert opinions. However, this argument was later proved that it is not always true. After that, Kitchenham describes about how to define and to use the quality instruments. Basically, checklists are used to assess quality in detailed. To construct the checklists, factors that could bias study results are considered.2.2.4 Data extraction and monitoringOnce the primary studies have been selected, the next step is to extract the relevant information. The extraction process should be performed as defined in review protocol which will describe the extraction forms used to collect the data from the filtered primary studies and also the proc edure of data extraction. Kitchenham discusses what should be contained in the data collection form. Not only the information to facilitate the answer of review question and the criteria for quality assessment are included, hardly also the basic information such as name of reviewer, date of performing data extraction and publication detail must be given. Importantly, the extraction form must be piloted before implementation. Kitchenham suggests that there should be two or more researchers perform data extraction independently. And they have to set agreements either by consensus or by using additional researchers to resolve disagreements on the data. If each paper cannot be assessed by at least two researchers, some checking technique, such as random sample of primary studies, has to be employed to ensure that the data are extracted correctly.Monitoring data is also important to perform in this stage. Kitchenham notes that multiple publications of the same study should not be contai ned in the systematic review since it can lead to bias. It is sometimes needed to contact the authors to make sure if those publications refer to the same study or not and also to derive the required information if the data obtained from studies are missing or we need some unpublished data.2.2.5 Data synthesisData synthesis aims to gather and summarize the data extracted from selected primary studies. Same as other stages, the activities to be performed should be defined in the review protocol. Base on Kitchenham research on various options of data combination from several types of studies, sensitivity analyses is suggested to perform to find out the impacts on the synthesis results where some studies are higher quality than others.2.3 Reporting the reviewThe purpose of this last phase is to write the results of the review. The guidelines explain there are three main stages in this phase Specifying dissemination mechanism Formatting the main report Evaluating the reportThe final rep ort does not only include the answers to the intend research questions, but it also need to specify the dissemination strategy so that the researcher can expose the result efficiently. Kitchenham presents seven mechanisms to disseminate the systematic reviews results. They area) Academic journals and/or conferencesb) Practitioner-oriented journals and/or magazinesc) Press Releases to the popular and specialist pressd) Short summary leafletse) Postersf) Web pagesg) Direct communication to affected bodiesBasically, the results are reported in two formulas in a technical report or in a conference or journal paper. After report the reports, it is necessary to perform evaluation. Kitchenham discusses evaluation technique for each type of reports. One effective technique is organizing a peer review. The structure and contents of report document can be seen in the Kitchenhams guideline 1.3. Improvement suggestions on Systematic review guidelinesThis section aims to present limitations an d to provide suggestion for improvement on each step of the systematic review guidelines on software engineering. The recommendations are collected base on lessons learned and experiences from various articles which utilize systematic review as literature review technique.The study reported in 7 reveals that one of the significant problems of publishing low quality systematic review is that some sight conducting systematic review do not understand exactly what systematic review is and how to perform it. So, they end up with having no obvious research questions, explicit search strategy and so on.The subsequent suggestions should be always kept in mind before starting and while performing the systematic review Thoroughly review and study the guidelines, e.g. Kitchenhams guidelines. Review several SLR examples and experiences to help you understand the process. Make sure you understand each step of activities and be sound for everything you do. Record the decisions made during condu cting the systematic review as much as possible since this information will be needed for writing the final report. accord to 2, they suggest to perform training on systematic reviewing in the very first step in order to make the reviewers familiar with the specific terms in the area where they will conduct systematic review. Furthermore, this helps the reviewers get better understanding about the review process and activities.3.1 Planning the reviewDuring this phase, the main activities to be performed are specifying the purpose of conducting a systematic review, formulating research questions, developing and evaluating a review protocol.3.1.1 Identification of the need for a reviewIn order to identify the bring statement of the objective of the review, the researchers should use the checklist to help pointing out the reasons and ensure their needs. Regarding this issue, there are several useful checklists provided in Kitchenhams guideline 1.In addition to identify rationale of th e review, Staples and Niazi 3 collected information from case studies, surveys, and reports to check if the intended research questions are possible to be answered by systematic review. This is founded very useful since they come to know what is common and uncommon in the research questions. Sometimes people thought the question is normal and feasible to be answered by other researches but in the later phases when they try searching for the related literature, they found that their question is very uncommon and then they have to discard those questions.Another crucial point is that the researchers should try to identify the existing systematic review related to their topics of interested in order to avoid conducting a duplicate review. Nevertheless, it is quite an difficult to find published systematic review in the area of software engineering comparing with medicine. Since there is no powerful scientific database which collects systematic reviews of studies related to software en gineering empirical studies, like the Cochrane (www.cochrane.com) which stores a large amount of systematic reviews of medical research. Although there are currently several go providing access to sources of software engineering publications, there are still many restrictions of those publications as followings 17 The available studies is limited and disintegrated properly since many researchers in this field are focused on their own style to generate result rather than structural review process. It is difficult to combine the result of software engineering review because the quality of review is so variable with no agreed standard for systematic review for this field. There is no guideline that is well-accepted, though some guideline has been proposed but it neither addresses all necessary topics nor provided sufficient detail.3.1.2 Specifying the research question(s)As mentioned before, the research questions are specified as part of the review protocol and will be used to constr uct the search string for searching related primary studies. Basically they will be revised repeatedly during piloting the review protocol and should not be changed when the protocol is committed.The most important point when formulate the research questions is to make them as obvious and concrete as possible. Other than structuring the question by using the PICOC criteria which is shown in 1, it is essential to specify rationale to formulate a particular question. Tabulate the question and purpose such in table 1 could be helpful.Table 1. Research questionResearch question PurposeRQ1RQ2Brereton et al 4 recommend that during protocol construction, researchers should anticipate to refine their research question both for increasing their understanding and making the automated search more effective. There are several systematic reviews, for example the Systematic literature review of guidelines for conducting systematic literature reviews in 4, which firstly define a few research quest ions. Later on, after investigation some information sources, those questions are extended in more detailed questionsRegarding 5, the research questions are not only the questions that needed to be answer by the review, but also the question providing some idea in the area of such a topic for better comprehension. This is also confirmed by Staples and Niazi 3 since their research questions are part of a larger research project. By performing the systematic review on these questions will help them understand better in the project background. By selecting clear and narrow research questions, it helps confining the scope of a systematic literature.4 proposes another method that may help to scope the research question. That is a systematic pre-review social function study. The idea is to map out sorts of studies relating the systematic review question have been conducted. The mapping process can be considered as a quick data extraction but the studies described are not very details. Th e further information about mapping study can be found at 12.3.1.3 Developing a review protocolAs explained in the section 2, the protocol provides information of the plan for conduction the review, including, for example, the procedure to be performed, the search strategy for selecting primary studies, the allocation of reviewers to some specific activities and the quality assessment criteria for evaluate primary studies. Lacking of a protocol, some process such as the selection of primary studies or data analysis and synthesis may be motivated by researcher bias 14. And because one of the key features of the systematic review is repeatable, a well-documented review protocol is needed to guarantee the reproducibility of the review.Many experiences on systematic review show that developing a review protocol is an iteration process that needs several revisions to get the complete protocol. Thus, the researchers should expect protocol changes, take a long time and allot appropriate ti me for it.Brereton et al 4 suggests all members in systematic review team should participate actively in constructing the review protocol, in which helps all of them get insights about the protocol and understand the process of data extraction.Additionally, piloting the review protocol is highly suggested to be performed. Not only because it supports discovering misunderstandings and mistakes in the data extraction and aggregation process, but also it may specify that the researchers need to change the method planned to deal with the research questions.As mentioned above, the search strategy must be documented in the protocol, enabling the reader of a review to evaluate how accurate and complete this is. However, due to the restriction of existing software engineering search engines that are not well-supported systematic reviews like in medicine, the software engineers should conduct resource-dependent searches. In particular, they might have to use different search string for each searchable sources which have different form interfaces and search syntaxes 4.The followings are recommendations to improve the search strategy. Searches should be performed to title as well as regard 6. However, before making a decision to accept or reject a primary study, searching on summary and content is founded very useful. Since mostly in software engineering, the abstract and titles are not much indicative 6 and not so dependable for primary studies selection 4. Any search strategy should be made up of multiple keywords, and it is essential to use a various combinations of terms to get the very successful search 6. Using more standardized vocabulary will promote the search results 6. For some software engineering topics, the publications in related fields should be searched as well. i.e. information systems, psychology, economics, quality, artificial intelligence 6 To get the most out of relevant publications, synonyms of the main search term should be used for search 6. It is likely that adding other more general terms to synonyms of the key search term detects more relevant studies. However, this tends to increase the number of irrelevant articles as well. So, it should only be used when there are a number of systematic review resources available in detecting and rejecting irrelevant articles 6 It is not necessary to search on all the search fields because it is not considerably benefits the result and also requires a big effort 6. Construct search strings using Boolean AND to link the key terms and OR to group synonyms 4. An example could be the following(experiment OR empirical study) AND(software cost estimation OR software effort estimation) Try searching from synonyms to the term representing the study type need to search 6. Include search fields that typically contain the key terms of the study type such as title and abstract 6. To get the key search terms. ,beak down the research questions into single words pertaining to the types of study t hat will help answer the question, technology of interested, and the reaction test 4. After deriving the key search terms, using various combinations of those terms to perform trial searches 6. Specifying the year of the paper first published in the search string can help lessen the amount of irrelevant articles 4. Be careful when using the Basic or Advances search forms because some search engines may produce different results even though the key searched terms are the s