PRINCIPLES OF MEASUREMENT IN NURSING EDUCATION RESEARCH
- Research in nursing education improves patient outcomes and benefits society.
- Publishing research findings helps nursing academics to secure tenure, gain funding for future research, and gain national recognition for their accomplishments.
- For research to effectively improve patient care and support nursing academics, the research must have sufficient reliability and validity so that the results can be trusted.
RELIABILITY
What is reliability? For any measurement, the score recorded by the observer is a combination of the true score that perfectly represents the subject and the error score that is caused by internal and external influences (Salkind, 2012). Reliability of a research method means that the same outcome is reached when it measures the same thing more than once (Salkind, 2012). In other words, if a measurement tool has high reliability, it keeps the error score at or below a minimum acceptable level.
Inter-Rater Reliability: This type of reliability is needed for measurement tools that will be used by more than one observer. To ensure that differences between subjects are due to the subjects themselves, and not to differences between the observers (Salkind, 2012). Inter-rater reliability is determined by having different observers rate the same subjects and then compare the percentage of scores that are in agreement (Salkind, 2012).
Test-Retest Reliability: This type of reliability determines the stability of a measurement tool over time (Salkind, 2012). In order to ensure that changes over time are due to changes in the subject, and not the instrument, the same instrument is administered to the same subject twice with a reasonable period of time in between (Houser, 2008).
One question in test-retest reliability is determining what is a reasonable period of time. There is no set standard; rather it is driven by the purpose of the research and what time period is seen as the most useful (Salkind, 2012).
Another question is determining the degree of reliability. Some tools are physical objects, such as thermometers and electrocardiograms that can be calibrated against a known standard. For psychosocial tools used in pretest/posttest research designs, the desired correlation coefficient for test-retest reliability is 0.5 or greater (Shultz & Whitney, 2005).
Inter-Rater Reliability: This type of reliability is needed for measurement tools that will be used by more than one observer. To ensure that differences between subjects are due to the subjects themselves, and not to differences between the observers (Salkind, 2012). Inter-rater reliability is determined by having different observers rate the same subjects and then compare the percentage of scores that are in agreement (Salkind, 2012).
Test-Retest Reliability: This type of reliability determines the stability of a measurement tool over time (Salkind, 2012). In order to ensure that changes over time are due to changes in the subject, and not the instrument, the same instrument is administered to the same subject twice with a reasonable period of time in between (Houser, 2008).
One question in test-retest reliability is determining what is a reasonable period of time. There is no set standard; rather it is driven by the purpose of the research and what time period is seen as the most useful (Salkind, 2012).
Another question is determining the degree of reliability. Some tools are physical objects, such as thermometers and electrocardiograms that can be calibrated against a known standard. For psychosocial tools used in pretest/posttest research designs, the desired correlation coefficient for test-retest reliability is 0.5 or greater (Shultz & Whitney, 2005).
VALIDITY
What is validity? For any research method, validity means that it is actually measuring what it is intended to measure (Salkind, 2012). It is important to keep in mind that validity refers to results of the measurement tool, but not the tool itself (Salkind, 2012). Also, the validity of the test results is interpreted within the context of the test itself and the environment in which it was administered (Salkind, 2012).
Construct validity: This type of validity determines the degree of correlation between the test scores and the underlying theory or model that explains what those scores mean (Salkind, 2012). A high degree of construct validity means that the measurement of a variable accurately captures the conceptual foundation of that variable (Houser, 2008). By linking a research study to an established conceptual model or to established theory, it is easier for researcher to establish construct validity of the research method.
Content validity: This type of validity determines to degree to which the measurement tool represents the entire domain of possible items under investigation (Salkind, 2012). Content validity isn’t measured with a coefficient like with reliability. Rather, content validity is a subjective judgment of whether or not a measurement makes sense (Houser, 2008). But not just anyone can make that subjective judgment. The best person to determine content validity is an expert in the field that this being investigated (Houser, 2008).
Peer review improves validity: Having an established expert review research design and tools ensures that the research is strong in terms of content, presentation, and contribution to nursing knowledge (Jasper et al, 2014). Effective peer reviewers should make objective observations on research design and methods, significance in terms of current practice, and clarity of presentation (Henly & Doughty, 2009). Most high-quality nursing journals require a minimum of two peer reviews by recognized experts in the content area (Jackson et al, 2011).
Construct validity: This type of validity determines the degree of correlation between the test scores and the underlying theory or model that explains what those scores mean (Salkind, 2012). A high degree of construct validity means that the measurement of a variable accurately captures the conceptual foundation of that variable (Houser, 2008). By linking a research study to an established conceptual model or to established theory, it is easier for researcher to establish construct validity of the research method.
Content validity: This type of validity determines to degree to which the measurement tool represents the entire domain of possible items under investigation (Salkind, 2012). Content validity isn’t measured with a coefficient like with reliability. Rather, content validity is a subjective judgment of whether or not a measurement makes sense (Houser, 2008). But not just anyone can make that subjective judgment. The best person to determine content validity is an expert in the field that this being investigated (Houser, 2008).
Peer review improves validity: Having an established expert review research design and tools ensures that the research is strong in terms of content, presentation, and contribution to nursing knowledge (Jasper et al, 2014). Effective peer reviewers should make objective observations on research design and methods, significance in terms of current practice, and clarity of presentation (Henly & Doughty, 2009). Most high-quality nursing journals require a minimum of two peer reviews by recognized experts in the content area (Jackson et al, 2011).
Ready to Learn More?
Click on the button to continue to the next page. |
REFERENCES
Henly, S. J., & Doughty, M. C. (2009). Quality of manuscript reviews in nursing research. Nursing Outlook
Houser, J. (2008). Precision, reliability, and validity: Essential elements of measurement in nursing research. Journal for Specialists in Pediatric Nursing, 13(4), 297-299.
Jackson, J. L, Srinivasan, M., Rea, J., Fletcher, K. E., & Kravitz, R. L. (2011). The validity of peer review in a general medicine journal. PLoS ONE, 6, e22475.
Jasper, M., Vaisomoradi, M., Bondas, T., & Turunen, H. (2014). Validity and reliability of the scientific review process in nursing journals: Time for a rethink? Nursing Inquiry, 21, 92-100.
Salkind, N. J. (2012). Exploring Research, (8th ed.). Upper Saddle River, NJ: Pearson Education, Inc.
Schultz, K. & Whitney, D. (2005). Measurement theory in action. Thousand Oaks, CA: Sage Publications.
Houser, J. (2008). Precision, reliability, and validity: Essential elements of measurement in nursing research. Journal for Specialists in Pediatric Nursing, 13(4), 297-299.
Jackson, J. L, Srinivasan, M., Rea, J., Fletcher, K. E., & Kravitz, R. L. (2011). The validity of peer review in a general medicine journal. PLoS ONE, 6, e22475.
Jasper, M., Vaisomoradi, M., Bondas, T., & Turunen, H. (2014). Validity and reliability of the scientific review process in nursing journals: Time for a rethink? Nursing Inquiry, 21, 92-100.
Salkind, N. J. (2012). Exploring Research, (8th ed.). Upper Saddle River, NJ: Pearson Education, Inc.
Schultz, K. & Whitney, D. (2005). Measurement theory in action. Thousand Oaks, CA: Sage Publications.