Human Factors in Software Engineering

There are many human factors involved in Software Engineering. It is reasonable to say so because software engineering is an essentially human centered activity.

In this paper, the human factors influencing software development process are named and mentioned.

Human factors are also covered in the topics of Management. Many human factors like Interpersonal Communication, Leadership, etc(I can recall from my Management class) are covered in Management classes. So, I will be dealing with some management concepts too in this post.

In fact, Software Engineering and Management has given birth to a new field called: Software Project Management.

Following are some concepts covered under both Management and Software Engineering.

- Cost Estimation
- Risk Analysis/Risk Management
- Activity Planning
- Resource Management
- People Management

Here I mention various phases in Software Development Life cycle and how they relate to various human factors:

Requirement Engineering

It involves the process of gathering information.

There are many human factors in RE.

- Multi-languages
- Culture
- Stakeholder
- Time zone boundaries

Design


Design is also related to many human factors.

Implementation


Apart from technical skills like knowledge of the code.
Implementation is also affected by characteristics of the programmer.

Testing


I have written a whole post describing how testing is influenced by human factors. You can read it here:
http://softwarequality-cognitivepsyc.blogspot.com/2010/10/psychology-in-software-testing.html

Mantainance


At this time, the project is more related to human issues.

Writing Technical Document


Writing technical documents is also greatly influenced by human factors.

For example, there are courses on 'International Technical Writing', where they take into account the various human factors like language differences, cultural differences while preparing technical documents.

References:

http://portal.acm.org/citation.cfm?id=3139.3140

Application of Cognitive science in HCI

Why does HCI need cognitive science?


In order to understand the user, it is useful to have a firm grounding in the cognitive processes that the user undergoes


The user perceives information, uses knowledge, and makes decisions, all of which are the domains of cognitive science


Understanding the user allows the HCI researcher to predict and explain the interaction that occurs between the user and the computer system


Without such an understanding, it is difficult to facilitate optimized interaction in the design of software.

There are many models in Cognitive Science like GOMS, MHP which can be applied in interface design. To understand more about these models read my other posts.

This post just serves as an introduction to guide the user to read other blog posts
which is :


http://softwarequality-cognitivepsyc.blogspot.com/2010/10/perceptual-cognitive-and-motor-task.html

8 Golden Rules of Interface Design

Any talk about interface design will not be complete unless a mention is made to the "8 golden rules of interface design". These rules are very important for designing an interface.

8 Golden Rules of Interface Design


We borrow the concept of using patterns from architecture. In architecture, we use design patterns for the construction of buildings. Wisdom is incorporated in these designs so other people can use them.

We also have the concept of architecture in computer science. Design patterns are used in computer science to solve the problem of usability and make the interface easier to use by the user and to increase user satisfaction with the interface.

                                          Ben Shneiderman, at University of Maryland lecturing
                                (Image taken from http://en.wikipedia.org/wiki/File:Ben_Shneiderman_Lecturing.jpg)



Recently, A computer scientist names Ben Shneiderman gave "8 golden rules of interface design". These rules were based on his research and empirical data collected through various experiments.

Here are the 8 rules:

1. Strive for Consistency
2. Cater to Universal Usability
3. Offer Informative feedback
4. Design Dialogs to yield closure
5. Prevent Errors
6. Permit easy reversal of actions
7. Support internal locus of control
8. Reduce short term memory

I have personally used these principles in the softwares that I have developed and found these immensely useful.

Let's discuss them one by one along with the insights I gained from these.

1. Strive for Consistency

Image taken from http://www.rockettinteractive.com/images/pic-consistency.jpg

Many forms of consistency:-

- Consistent sequences of actions should be required in similar situations
- Identical terminology should be used in prompts, menus, and help screens
- consistent color, layout, capitalization, fonts and so on should be employed throughout.

If menus, prompts, help screens are consistent, the user can quickly make out what is to be done in the next step.

2. Cater to Universal Usability

                                (Image taken from http://www.maps-continents.com/maps/globe-africa-countries.jpg)

A good interface should cater to universal usability.

There may be diverse users using your interface. You need to address the needs of diverse users.

For example, If you are creating a website, it is a good idea to create a website in such as way that it addresses both the low-bandwidth and high-bandwidth users.  Many good websites do this these days.

Sometimes implementing universal usability can be challenging because we need more resources to address a broad range of users.

3. Offer Informative Feedback


Offering informative feedback to the users for the actions they do gives the users confidence that they are proceeding in the right direction.

4. Design dialog to yield closure


Sequences of actions should be organized into groups with a beginning, middle and end.

Image taken from www.amazon.com

For example, when you checkout at amazon it takes you step by step through a sequence of actions. For example, you sign in, select payment option, place order, etc.

5. Prevent Errors


A good interface reduces errors , there can be many ways of doing so:

- Use selection instead of freestyle typing

A person may commit a typo while freestyle typing. So, selection is a good way of reducing that error.

 - Automatic completion

 Showing Google Autocomplete(http://alias-i.com/lingpipe/demos/tutorial/querySpellChecker/google-auto-complete.png)

Many search engines like Google use technology called AJAX which is used for automatic completion as we are typing in the query.

- Make error messages specific and positive in tone

An error messages should be positive in tone and most importantly specific. A specific message let's t he user know what he needs to do.

6. Permit Easy Reversal of actions


Permitting easy reversal of actions is a great way to :

- encourage user to explore new options
- Relieve anxiety

7. Support Internal Locus of Control

Every one like control in their hand!

User satisfaction is high when they feel that they are in control of the system
To do so:

- Avoid surprising interface actions, tedious data entries, etc.

User should feel that they are controlling the system.

8. Reduce short-term memory load


Human processing is limited in short term memory.
Therefore:

- Keep display simple
- Consolidate multiple-page displays , reduce windows-motion frequency.

References:

http://konigi.com/notebook/shneidermans-eight-golden-rules-interface-design
http://en.wikipedia.org/wiki/Universal_usability

Perceptual , Cognitive and Motor task theories

In my last blog, I described the Model Human Processor and made reference to the GOMS model.
Here I will explain the GOMS model.
I will discuss the various perceptual, Cognitive and motor task theories related to HCI.

There are different perceptual, cognitive ad motor task theories applied to HCI

Perceptual Theories

Perceptual theories deals with how we perceive the world around us. The sensory system, for example, the eye in the case of visual perception, incorporate information about the natural world into their design and "relate" incoming stimuli to this information, rather than analyzing sensory stimulation into its components.

Perceptual theories are used for -

- predicting reading times for free text, lists and formatted displays

Cognitive Theories


Cognitive theories are based on the assumption that human beings are logical beings and it analyzes the thought processes of human beings.

Motor-task performance times theories


After thinking, the next step is action or motor tasks. Motor-task performance theories can be used for:-

- Predicting key-stroking or pointing times

Models


There are various models in cognitive science that pertains to human-computer interaction.For example:

1. Stages of Action Model
2. Goals, Operators, Methods, and Selection rules(GOMS) Model
3. Widget-level theory
4. Object-Action Interface Model

Stages of Action Model


Norman's seven stages of action :


1. Forming the goal
2. Forming the intention
3. Specifying the action
4. Executing the action
5. Perceiving the system state
6. Interpreting the system state
7. Evaluating the outcome

Norman seven stages of actions can be broken into two loops:


1. Execution Loop
2. Evaluation Loop

Execution Loop


Execution loop means when the execution begins.

Evaluation Loop


Evaluation loop involves evaluation and perception.

The following stages of the seven stages are a part of Execution Loop :

2. Forming the intention
3. Specifying the action
4. Executing the action

The following stages of the seven stages are a part of Evaluation Loop :

5. Perceiving the system state

6. Interpreting the system state

7. Evaluating the outcome

Image from http://www.cc.gatech.edu/classes/cs6751_97_fall/projects/spin/mayc/action.gif

Let's Understand this by an example:

Let's say you are working on your laptop computer and your  battery is getting lower and lower. So, you need to charge your battery using an AC adapter.

Execution phase example:

So, the GOAL is to charge the battery of the adapter.

To accomplish this goal, you need to stretch your hands in a certain way and plug in the adapter.  So, the GOAL has translated into an INTENTION which has translated into an ACTION SEQUENCE.

So, first the goal is formed. Then, there is an intention. Then, the intention is translated into an action sequence. 

Nothing happens until EXECUTION, which means that the action is executed in the real world

Evaluation Loop Example:

After we have turned on the charger, we Evaluate how it is affecting the system and make an evaluation.

GOMS Model

Goals : what the user has to accomplish

Operator: An action performed in the service of the goal

Methods: Sequences of Operators and sub-goal invocations that accomplish a goal.

Selection rules: If there is more than one method applicable to  the goal, selecting rules are necessary to represent the knowledge of which method is to be applied.

Using these elementary actions as a framework an interface can be studies. There are several different GOMS variations which allow for different aspects of an interface to be accurately studied and predicted.

Widget-Level Theory

WLT deals with higher level components that are self-contained and reuseable

Widgets are interface components, such as buttons, selection lists, radio buttons, sliders.

Object/Action Interface Model

"Direct Manipulation" is an important concept in interface design.

The principle of direct manipulation contends that having real-world metaphors for objects and actions can make it easier for a user to learn and use an interface.

There are many applications of direct manipulation in interface design. For example, computer graphics, 3D animations are based on the principle of direct manipulation.

The Object/Action interface model is based on the principle of direct manipulation.

Object-action design:

1. Understand the task

           - real-world objects

           - action applied to those objects

2. Create metaphoric representations of interface objects and actions

3. designer makes interface actions visible to the users

In an object-action model, the user first selects an object and then performs an action.

Example of object-action model: If you have to copy a file from one folder to another in a Windows-based system. You first select the object(a file in this case) and then perform the action(i.e. copy)

I will conclude this page here. The models discussed here are very important to the interface design. The applications of these models will be shown in my upcoming posts.


References:


http://www.cs.umd.edu/class/fall2002/cmsc838s/tichi/oai.html

http://en.wikipedia.org/wiki/GOMS

http://en.wikipedia.org/wiki/Seven_stages_of_action

http://www.d.umn.edu/~ddunham/cs5551f08/shneidppt/DTUI_chap02p.ppt.

Psychology in Software Testing

You are my son!
Yesterday, I was watching a movie - Wednesday in which they showed how a person developed attachment for his car and treated it like his son. He literally called it his son and cried when it broke.
The same applies to softwares.
After all, we are all human beings influenced by emotions.
A farmer develops some attachment to his field, a horse-owner becomes proud of his horse. Similarly, Software developers develop attachment for their program after they have worked on it for a considerable time. This is something I will call as

Psychology in Software Testing

Suppose you have worked on a code for months together  and then you have to test it. Due to psychological factors, you will be more inclined to prove your software is working right than wrong.
This is a proven fact.
Due to this reason, many software companies have a completely different unit for software testing.


Do good and get evil!


The biblical principle of doing good and getting good does not apply to the software testing.

Software testing is screws where you have to break someone's code. Of course, this is the traditional view only. More recently, there are other types of tests like stress test which does not necessarily involve breaking someone's code.

In a dilemma


Suppose a situation where you are a software tester and you have to test the program of a colleague friend.
Now you are in a dilemma, if you break the code, the developer may be screwed for not writing good enough code . On the other hand, if you do not break the code, you might be screwed for not writing good enough test cases.

Creatively Destructive


Being good testers might be innate to some creatures. It needs "creative destructiveness" to be a good tester.  Some people may be innately good at it while other not.
There is some support to this argument. For example, Meyers concluded that :

"The psychology of the persons carrying out the testing will have an impact on the testing process [Meyer 1979]."

Some Characteristics of  a good test:

Like good people have some good characteristics, so do good tests.

Here are some characteristics of a good test:


(from page 432, Software Engineering : A Practitioner's Approach, Sixth Edition")


1. A good test has a high probability of finding error.
2. A good test is not redundant.

There is no point in doing a test that has same purpose as another test

3. A test that has the highest likelihood of uncovering a whole class of errors should be used.

4. Make a good test, neither too complex nor too simple

A test which is too complex can "mask" errors.

It is important to note that : A good test shows the presence of bugs but does not guarantee their absence.


Like test cases, there are some characterstics you can develop to become a "good tester" :



Keen Observation
Detective Skills
Destructive Creativity
Understanding Product as integration of its parts
Customer Oriented Perspective
Cynical but Affable Attitude
Organized, Flexible & Patience at job
Objective & Neutral attitude

The above mentioned characteristics can prove immensely helpful apart from technical skills.

References:


1. Software Engineering : A Practitioner's approach , Sixth Edition
2. http://www.codeproject.com/KB/bugs/Pratap.aspx
3. http://www.viarob.com/my/page/The_Psychology_of_Software_Testing

Important concepts in Human-Computer Interaction

This blog explains some important factors in human-computer interaction especially those that relate to vision, perception, psychology, etc.

Human Computer Interaction Concepts

Aimed movements -- pointing at a target-- are a series of micro-movements that "zero in" on the target as visual feedbacks.

Movement time depends on the number of micro-movements , which depend on the target size(S) and distance(D).

Far targets relatively fast

Small targets much slower

This is known as Fitt's Law:

Fitt's law states that the time for aimed movements (such as mouse cursoring) is proportional to the logarithm of the ratio of distance over target size. This means that the time increases when the target gets further away or smaller.

Here is an interesting VIDEO ABOUT FITT'S LAW:

http://www.youtube.com/watch?v=EpBa3VzD5xY

In other words, time depends upon target size(S) and distance(D).

- Far targets relatively fast

- Small targets much slower

  

Cursor Key vs. The Mouse

When analyzing cursor key vs. the mouse, the cursor key is effective in some cases. For example, for placing the cursor between the two periods.

In accordance with fitt's law, mouse can be more effective in showing things on the screen.

The mouse follows the fitt's law:

Or rather PROPERLY DESIGNED mouse follows the Fitt's law

- Extremely efficient for long moves

- Not much longer to reach the cursor keys

- But small targets can be a problem

Other devices are slower because time governed differently

- E.g. Cursor key linear with total of x and y distance.

- Joysticks can be very different, in some cases quite difficult to learn and to use- depending upon control regime

Mouse is a good way to point, but keyboard are faster if only a few keystrokes needed.

Keyboards

- Hand entry of characters is very slow

- Keyboards are much faster after practice

- Pen-based interfaces are not fast but have other advantages

Output Basics: Human Vision

It is interesting to note that how the output should be displayed is affected by human vision.

Human vision uses two receptor systems:-

- High resolution color system

- Receptors("Cones")

- Requires higher light levels(e.g. normal daylight(official light)

- Receptors from three color ranges : red, blue, green

Sensitive Monochrome Light

Receptors("rods") 

Poor resolution - relatively bw reception density

Extremely sensitive but not involved in normal lighting

Basic Principles of Display Design:

What NOT to do: Exclusive focus on appearance

Be concerned only with what the screens look like; screen layouts, color icons, attractiveness

What to do: Present the right information in the right way on the right time

Choose the right information:

- What decisions will the user make at each point in the task?

- What information will the user need to make those decisions?

Pick a good representation of the Information

- Different ways of representing the same abstract information can have huge effects = the "Representation " effect

- Attempt to present all relevant information simultaneously

because : Eye can move in milliseconds from one place to another

This means that:

-> Visual search of even a complex display can be extremely fast compared to bringing up additional screens.

- With good visual layout, displays can deliver much more information than generally delivered.

Icons vs. Words as Display Objects

Misguided Style: Use nifty icons , not boring words

- Icons are fun, decorative BUT NOT usually helpful.

Icons are definitively better mouse targets than words

- Square shape gives more target area

- Words tend to be long and thin ; often provide a small target

Icons are often arbitrary and meaningless as compared to typical words for computer objects

- Icons can be hard to recognize compared to words

especially, if the item concepts are abstract

WHY MAKE USER MEMORIZE MEANINGLESS SYMBOLS?

Throughout his life, the user has used words for information processing

- Words can be better than icons for doing visual search.

- Icons are generally too complex to guide visual search.

Role of Psychology in Human-Computer Interaction -- II

Role of Psychology in Human- Computer Interaction

In my last blog I described the MHP( Model Human Processor).

Now, I will give some examples of the the same.

Example 1: Simple Reaction Time

Production rule "Programming" for simple reaction.

Pattern-action pair :

patterns are tested in parallel 

Actions are performed serially

Push the button as soon as you see the light on display.

Example : Simple Reaction

Cognitive processor uses rule to decide to make response

Motor processor responds to command in working memory.

The MHP is a small picture.

The MHP is an example of how many details can be organized into an architecture for predicting and explaining cognition and performance relevant to HCI

Another example is :: 

GOMS

GOMS is a simpler version of the same approach

(I will explain GOMS in some of my upcoming blog)

Role of Psychology in Human-Computer Interaction -- I

Role of Psychology in Human- Computer Interaction

It intrigues me to notice how various branches of knowledge like psychology, philosophy, cognitive science and human computer interaction are inter-related.

The inter-related nature of these fields gives us an insight to learn from one field and apply the learning to the other. The same holds true for the field to human-computer interaction.

Human-computer interaction is a field of knowledge that deals with the study of interaction of humans and computers. This field is very much related to software engineering and involves techniques by which the softwares can have a good interface meeting user demands.

Human computer interaction is generally viewed as dealing with the development of software. However, this field also deals with the development of hardware devices.

The important part of HCI is to ensure user satisfaction.

The term "User Satisfaction" can be a broad term. It can have different meanings for different people. For example, a lay person who is not very efficient with the use of computers may be happy with an easy interface like the windows operating system. However, a more skilled user, who is an expert with UNIX commands, may like UNIX because it is more efficient. So, for a layperson user satisfaction means ease of use whereas for a skilled person user satisfaction means efficiency.

So, different users may have different expectations from a user interface and we must design a user interface so that it meets the demands of our target audience.

Psychology is of immense use when trying to accomplish this.

There are many ways in which psychology can contribute to effective interface design. For example:-

1. Using psychology, we can analyze the thought/cognitive process of the user and predict what user will do. Based on this prediction, we can make an effective user interface.

2. Human beings have certain capabilities and limitations which affect the way we use an interface. Psychology takes this into account and thus helps us to make effective interfaces. For example, we have been used to using words than symbols. So, our brain can process words more quickly than images. So, words are better used than icons at many places.

Our eyes too have limitations and certain capabilities which can be taken into account for designing interfaces. For example, Researches have shown that we can process more complex information presented in a visual diagram on one screen than if the same information is present on various screens.

Now, I will be narrowing down to my topic which deals with interlink b/w psychology and HCI.

I will be mentioning cognitive models later in this article . To understand them well, here are too terms from Philosophy class:

Declarative Knowledge:

- Knowledge of facts

- It has a list of inter-related facts or propositions

- It is stored in Long - Term Memory

- Flexibly used

- Easy to report

- Can be quickly acquired under certain conditions

Procedural Knowledge:

- Knowledge of what to do

- Represented as production rules:

IF THEN rules

- Stored in production memory

- Less Flexible

- Can't be reported

Here is how they apply to Human Computer Interaction:

Both declarative knowledge and procedural knowledge are relevant to interacting with a device

- The more knowledge required , the harder it is to learn and use a system

Ease of learning and ease of use go together.

Ease of use is an important factor in human computer interaction which is affected with declarative and procedural knowledge.

These two concepts (procedural and declarative knowledge will be used further in the article)

Imagine a scenario where I give you a new type of interface(something different from what you use regularly), say a new type of digital clock. Let's say your goal is to set an alarm in that clock.

In view of Rationality Principle, your attempt to accomplish the task will be affected by several things. For example:

- Characteristics of the task

Some tasks are more complex than others

- Actions possible in the task

- Inputs to the person

- Person's knowledge

The more knowledge you have of the system, the more likely you are to reach your goals/

- Person's processing limitations

Rationality Principle and Human Factors:-

- Basic goal of good system design is to alter the task, operators so that the human can be optimal in accomplishing goals in-spite of processing limitations.

So, Rationality Principle gives us a set of variables to work with. We can alter these variables to accomplish the goal of effective interface design.

Now, let's understand some concepts of Cognitive Science.

The Model Human Processor

Click Here to see a better image of MHP

In Cognitive Science, there are many information processing models . For example, GOMS, MHP. Model Human Processor is also one of the information processing model which explains how information is processed.

MHP is a simple information processing model of overall structure and mechanisms involved in human cognition.

Cognitive Processor takes information from the production memory and long-term memory and working memory.

Cognitive Processor take information from the production memory and the long-term memory and the working memory.

Cognitive Processor

- Production system model for cognitive processor:-

- "Programming" for cognitive processor consist of production rules:

IF(pattern) THEN (action)

IF(pattern) THEN (action)

Patterns tested in parallel , actions performed serially

Procedure or skills composed of a set of rules that fire in a proper sequence depending upon the situation

Works pretty well in detailed of skill and problem solving

Cognitive process "thinks" by manipulating working memory(WM)contents according to production rules.

- Working memory is the scratch pad memory

- Holds perceptual output, etc.

- In each cycle, contents of WM and LTM can trigger associated actions.

Actions can:

- Alter contents of VM

The Model Human Processor as explained in used in Psychology as well as other fields like Artificial Intelligence.

This MHP is especially useful in Artificial Intelligence.

In Artificial Intelligence, we have production memory and declarative memory and the working memory acts as a scratch paper for the cognitive processor.

This model is used in Artificial Intelligence design.

For example, If there is an artificial live chat agent on a website made using artificial intelligence, it uses information from Procedural Knowledge and Declarative Knowledge to act as a basis for working memory which is a scratch pad. For example, when an input is made to the system, the cognitive processor "thinks" by manipulating working memory(WM) contents according to production rule. So both procedural knowledge and declarative knowledge are used by working memory which directs output to the motor process.

Cont.. in next blog(  considering the increasing size of the document, I have broken it into two parts. the next part -- part II is in next blog)


References:

http://videolectures.net/chi08_kieras_phc/

Software Defect Reduction Top 10 list

One : Software defects are easier and less expensive to fix in an earlier stage than a later stage.

Finding and fixing a software problem after delivery is often 100 times more expensive than finding and fixing it during the design phase

In other words, the sooner you correct  a bug in a software development life cycle the better it is in terms of reducing cost and effort.

There are many software practices by which defects are likely to be found in earlier stages.

- Through requirement analysis and design
- Early verification and validation
- Upfront prototyping and simulation to avoid costly downstream fixes.

Guidelines for performing Systematic Literature review

DISCLAIMER : This information is not my own. I am just summarizing the "

Guidelines for performing Systematic Literature Reviews in Software Engineering by Barbara Kitchenham, Stuart Charters".

Purpose of this page

The information presented here are some basic guidelines for a systematic literature review.

Target Audience

It is aimed primarily at the researchers in software engineering community.

What is Systematic Literature Review?

-> Means of evaluating all the available research relevant to a particular research question.

Importance of Systematic Literature Review:

1. It identifies gaps in the current research and suggests further areas of investigation.

2. It provides reference upon which new research topics can be build upon.

Characteristic of a good systematic review:

1. It should be thorough and fair.

Researchers performing a systematic review should make sure that they report research that does not support their research hypothesis.

Important components of systematic literature review:

1. Systematic review starts by defining a review protocol that specifies the research question being addressed and the method that will be used.

2. Search strategies need to be developed to cover as much information as possible.

3. Search strategies need to be documented.

Nature of Software Engineering Research

Software engineers are human beings so it is more similar in experimentation to social sciences.

The Review Process

Systematic review has three phases:

1) Planning the review

2) Conducting the review

3) Reporting the review

These three stages may be in order or may follow an iteration.

The stages associated with planning the review are:

1. Identification of the need for a review

2. Commissioning a review.

3. Specify a research question

4. Develop a review protocol

5. Evaluate the review protocol

The stages associated with conducting the review are:

1. Identification of the research

2. Selection of primary study

3. Study quality assessment

4. Data extraction and monitoring

5. Data Synthesis

The stages associated with reporting a review are:

1. Specifying dissemination mechanisms

2. Formatting the main report

3. Evaluating the report

The Planning Stage

Why plan?

Planning is necessary to confirm the need of such a review.

The most important activities are::

1. Identify the questions that the review will intend to answers.

2. Producing a review process(plan) defining the basic review process.

Formulating Research Questions

- The search process must identify primary studies that address the research questions

- The data extraction process must extract the data items needed to answer the question.

- The data analysis process must synthesize the data in such a way that the questions can be answered.

A Right Research Question could be :

1. The one that identifies the discrepancies between commonly held beliefs and reality.

2. Is meaningful and important to both practitioners and researchers

3. Will lead to changed in the current practice or increase confidence in the value of current practice.

Review Protocols

Purpose of Review Protocols

Identify methods that can be used to understand a systematic literature review.

Reasons for having review protocols::

--> To prevent bias

A pre-defined protocol is needed to reduce the possibility of researcher's bias.

Otherwise, it is possible that the selection of studies may be driven by researcher's expectations.

Common components of a protocol:

1. Background. The rationale for the survey

2. The research question that the review is intending to answer

3. Strategies used for primary study

4. Study selection criterion

5. Study selection procedures

6. Study quality assessment. Checklist and procedures.

7.Data extraction stratergies

8. Synthesis of extracted data

9. Dissemination stratergy

10. Project time table

Evaluation of Review Protocol

PhD students should present their protocol to their supervisor for review and criticism.

Conducting a Review

Identification of Research

In systematic review, the research process should be rigorous. The researcher should find as many primary studies relating to the research question as possible using unbiased research stratergy.

Generating a search strategy

- Preliminary studies aimed at both identification and systematic review

- Trial searches using various combinations

- Consultation with expert in the field.

- Journals, reference list from review articles, internet can prove to be important sources of evidence

Avoiding Publication Bias

Publication bias means that positive results are more likely to be published than negative results. The concept of positive and negative results sometimes depend upon the viewpoint of the researcher.

Common scenarios of Publication bias

- Sometimes there is publication bias because failure to reject null hypothesis is considered less interesting than a experiment that is able to reject null hypothesis.

- Publication bias occurs when an influential company/group wants to support a particular method/technique.

Searches must be documented and a Bibliography must be maintained

Search process documentation should be fairly detailed so that the reader can go back.

Researchers should specify their rationale for :

- The digital library to be searched

- The journal of conference proceedings to be searched

Study Selection Process

Step 1: selection criteria should be interpreted liberally, so that unless a study identified by the electronic and hand searches can be clearly excluded based on title and abstract, a full copy should be obtained

Step 2: The next step is to apply inclusion/exclusion criterion based on:

- Journal , languages, author, etc.

Step 3: The third step in study selection could be detailed quality criterion.

It is a good idea to keep a list of excluded studies and reasons for exclusion.

Data Extraction

The objective of data extraction is to design data extraction forms to accurately record the information researchers obtained from the primary studies.

Design for Data Extraction forms

The data extraction forms must be designed to collect all the information needed to address review questions and the study quality criterion.

Contents of data collection form

In addition to the question needed to answer the review question and quality evaluation criterion, data collection forms should provide the following information including:-

- Name of the reviewer

- Date of data extraction

- Title, authors, journals

Reporting a Review

Specifying a Dissemination Strategy

The dissemination strategy , should be planned during the dissemination stage ( if any) preparing for systematic review protocol.

Formatting the review report

Systematic review can be reported in two formats:

- In a technical report

- In a journal style conference paper

Where to publish?

"Journal of Information and Software Technology" is a good place to publish literature review.