Quantitative Research.

Prof. Barrie Gunther.

The following is an overview of basic concepts within quantitative research, including hypothesis testing and sampling, reviews of survey research, quantitative content analyses, and experimental studies of media with analytical examples. It also offers a comparison of surveys and experiments, their strengths and weaknesses and a presentation of quantitative data analysis, including examples of statistical procedures.

Media and Communication research has a variety of different research techniques. These methodologies are the subject of much debate on which research perspective offers the most meaningful insights into the role and influence in society. Hypothetico-deductive approaches are concerned with proving or disproving a hypothesis and the establishment of theoretical explanations of relationships between individuals and media. These phenomena are explained in quantitative terms to demonstrate measurement of links between them.

Basic Concepts In Quantitative Research:

The first concept to be considered is that of the ‘variable’. This is an empirical representation of a concept or construct. A ‘concept’ represents an abstract idea that embodies the nature of observable phenomena. Constructs are a combination of concepts.

Variables With Attributes:

In the case of gender there are two attributes – male and female. Variables can be defined further in terms of their relationship to each other. Independent and dependent variables the former can be manipulated and the latter cannot. Researchers provide evidence that independent variables have a casual relationship with a particular dependent variable. Take for example the placement of a news story in a TV news bulletin and its effect on audience recall. In this scenario the placement of the news story (start, middle, or end of bulletin) will affect the recall of its content for the viewer. In this case the hypothesis is ‘Placement of a news story in a bulletin will dictate the recall of the viewer’ – the Independent variable is the positioning of the story and the dependent variable is the viewers recall.

Hypothesis Testing:

Quantitative research is concerned with cause-effect relationships. The hypothesis is the proposition to be tested. The hypothesis tests the co-relationship between independent and dependent variables. The researcher then sets out to discover if that prediction holds true. The outcome is a contribution to the growth of knowledge.

Reliability and Validity:

Reliability concerns the dependability and consistency of the relationship between the two variables at more than one point in time. Validity indicates whether a measure properly captures the meaning of the concept or construct it represents.

Validity Assessment:

This is done in several ways. For example in a literacy survey of a group of people the outcomes can be validated as follows:

a) Face validity i.e.; testing literacy and giving the subject something to read;

b) Predictive validity, which the reader can read to a predicted level;

c) Concurrent validity: subject can read to a previously determined standard;

d) Construct validity; reader is shown to be related to a variety of other established and previously verified measures.

Internal and External Validity:

Internal validity proves that the experiment has successfully measures what it set out to measure; external validity tests whether the results can be generalised to other situations or groups of people.

Levels Of Measurement:

There are four types of measurement in quantitative research and these are nominal, ordinal, interval, and ratio.

A) Nominal: any ‘object’, which is placed in this category, is deemed to be equivalent to any other.

B) Ordinal is a categorisation on a scale, for example, is the respondent lower class, middle class or upper class.

C) Interval: As with the temperature scale it rises in degrees with each degree equivalent in distance to any other.

D) Time, distance, and speed are examples of ratio scale. An object moving at ten miles an hour is moving twice as fast as one moving at five miles an hour.

Media Research and Quantitative Techniques:

These basic techniques are applied in quantitative research to measure media audiences, media content, and cause-effect relationships.

Searching For Media Effect Associations:

Survey Research:

Surveys are a major part of quantitative research that does not involve any manipulation of participants or their circumstances in advance. (i.e. Census) Surveys collect data after the fact. Surveys explore relationships between variables. Now every survey is a census and smaller surveys will seek the participation of samples of population.

Purposes of a Survey:

Descriptive; attempts to document current conditions, analytical; collects and examines data for relationships amongst variables in order to test a hypothesis.

Forms of Administration:

Surveys collect data through questionnaires or interviews.

The Issue of Sampling:

Participation in a survey should be representative of the total population from which they are drawn. Samples may be constructed on a probability or non-probability basis.

Non-Probability Sampling:

People are selected for study on the grounds that they are available, convenient to access and prepared to participate. The researcher has little control over who comes forward and consequently such samples are likely to be biased in their demography and psychological characteristics as compared to the population in general. Purposive sample is one, which respondents are selected according to a specific criterion and a ‘quota’ sample is where a selected amount of similar people are sampled.

Probability Sampling:

There are four kinds as follows: random sampling; whereby all population is represented by random selection; systematic random sampling whereby a criterion is fixed to select every nth person from a population; stratified random sampling; restrictions are placed upon the selection process although the fundamental element of randomness is retained; cluster sampling; involves a special case of stratification.

Time Span:

Surveys can be either ‘one off’ or ‘cross-sectional’ to obtain either opinions ‘today’ or opinions ‘today’ and at future dates.

Longitudinal Research:

Used for examining long-term relationships between selected variables and demands a collection of responses over time. There are three types of longitudinal research:

1. Trend studies; samples are selected and question and re-questioned over time to measure trends;

2. Cohort studies; focuses on the same subset of a population although samples may be different (for example, a survey questioning 10 year olds in an area about the introduction of TV and asking them 5 years later how it went);

3. Panel Studies; The collection of data over time from the same respondents.

Surveying Media Output:

‘Content Analysis’ was taken up by social scientists to monitor general social and economic trends. It can be used, for example, to measure public reaction to news content and is a research technique for the objective, systematic, and quantitative description of the manifest content of communication. Five main purposes of content analysis are;

1. Describing Patterns Or Trends In Media Portrayals.

2. Testing hypotheses about the policies or aims of media producers.

3. Comparing media content with real world indicators.

4. Assessing the representation of certain groups in society.

5. Drawing inferences about media effects.

Testing Causality Directly:

Experimental Research:

Experiments are also used in research. An experiment begins with a hypothesis about a likely outcome following an event that can be controlled or manipulated by the researcher. Participants are exposed to a particular media stimulus and then questioned. Participants are allocated to one of two groups – experimental group or control group. The former are exposed to the manipulated independent variable while the latter are not. The main advantage of experimental research is that it enables research to test for evidence of cause-effect relationships between variables.

Experimental Designs:

Classic experimental design; uses two groups and participants are randomly allocated and then tested.

Pre-experimental Designs:

A test is conducted without prior test to ascertain the knowledge of the participant in relation to the subject of the test;

Quasi-Experimental Designs:

Participants are randomly allocated to experimental and control groups but are tested only after the experimental manipulation has been implemented.

Factorial Designs:

Two or more independent variables are manipulated.

Repeated Measures Design:

A repeated measures design refers to studies in which the same outcomes are measured at two or more distinct times for each subject. For instance, repeated measures are collected in a longitudinal study in which change over time is assessed. Other studies compare the same measure under two or more different conditions. For instance, to test the effects of caffeine on cognitive function, a subject’s math ability might be tested once after they consume caffeine and another time when they consume a placebo.

Problems With Laboratory Research:

An important shortcoming with experiments stems from the conditions they create for examining links. Laboratory outcomes are not, nor will ever be, real life outcomes.

Surveys Or Experiments And How Do They Compare?:

Surveys and experiments are two forms of quantitative research concerned with aspects of media. The weaknesses of experiments are;

1. Their use of non-representative samples.

2. The degree of artificial control.

3. Their contrived nature.

4. Control difficulties.

The weaknesses of surveys are;

1. Their dependence on post – hoc.

2. Their use of verbal measures.

3. Their reporting only of degrees of association.

Handling Quantitative Data:

Surveys and experiments produce data, which needs to be analysed and interpreted. Data will accept or reject the hypothesis of the survey or experiment.

Describing Data:

Data is extracted from paper formats and entered into databases and tested. Errors in data transference can result in invalid results. Data can produce a ‘central tendency’ which can be defined in three ways; mode, median and mean. In mode sequence the surveyor which score (from one to 10 choices most people ticked 6) then the mode is ‘6’. The median score is the midpoint (from 1 to 10 choices most the middle is 5) then the ‘median’ is 5; The ‘mean’ score is the average of the total range of scores; (if the total scores are 1 to 5 then the total is 1+2+3+4+5 = 15 / 5 (5 being the total range) = 3 then the mean score is 3.

Testing Hypothesis:

The researcher is interested in whether two or more variables are associated in an unambiguous or significant way. The starting point is known as the ‘null hypotheses. There are two ways of testing the hypothesis and these are parametric and non-parametric tests. A statistical hypothesis test is a method of making decisions using data, whether from a controlled experiment or an observational study (not controlled). In statistics, a result is called statistically significant if it is unlikely to have occurred by chance alone, according to a pre-determined threshold probability, the significance level. The phrase “test of significance” was coined by Ronald Fisher: “Critical tests of this kind may be called tests of significance, and when such tests are available we may discover whether a second sample is or is not significantly different from the first.”

Confirmatory Data Analysis:

Hypothesis testing is sometimes called confirmatory data analysis, in contrast to exploratory data analysis. In frequency probability, these decisions are almost always made using null-hypothesis tests (i.e., tests that answer the question Assuming that the null hypothesis is true, what is the probability of observing a value for the test statistic that is at least as extreme as the value that was actually observed?) One use of hypothesis testing is deciding whether experimental results contain enough information to cast doubt on conventional wisdom. A result that was found to be statistically significant is also called a positive result; conversely, a result that is not unlikely under the null hypothesis is called a negative result or a null result. Statistical hypothesis testing is a key technique of frequentist statistical inference. The Bayesian approach to hypothesis testing is to base rejection of the hypothesis on the posterior probability. Other approaches to reaching a decision based on data are available via decision theory and optimal decisions. The critical region of a hypothesis test is the set of all outcomes, which cause the null hypothesis to be rejected in favour of the alternative hypothesis. The critical region is usually denoted by the letter C.

Conclusion:

These methodologies are used to study media audiences and content and they do this by examining variables or cause-effect relationships. This is achieved by applying the hypothetico-deductive approach (hypothetical situations are proposed and then accepted or rejected through the collection and analysis of data.) Phenomena is reduced to numerical codes which in turn can categorise and rank phenomena and be used to measure relationships among phenomena and to establish casual relationships. Quantitative research aims to enhance knowledge by demonstrating links between phenomena and the universality of such relations. However, quantitative studies (as with all research) should be carefully scrutinised for their methodological limitations and the quality of their data before any weight or credibility is given to the findings.

Primary Source

Prof. Barrie Gunther.

Media Research Techniques

Click On Image To Learn More.