Validity and Reliability
Written by Ari Julianto
The two most important and fundamental characteristics of any measurement procedure in research are reliability and validity. These two principles in research are as followsI. Validity
Validity is defined as the extent to which the instrument measures what it purports to measure. For example, a test that is used to screen applicants for a job is valid if its scores are directly related to future job performance.
Validity is arguably the most important criteria for the quality of a test. The term validity refers to whether or not the test measures what it claims to measure. On a test with high validity the items will be closely linked to the test’s intended focus. For many certification and licensure tests this means that the items will be highly related to a specific job or occupation.
If a test has poor validity then it does not measure the job-related content and competencies it ought to. When this is the case, there is no justification for using the test results for their intended purpose. There are several ways to estimate the validity of a test including content validity, concurrent validity, and predictive validity. The face validity of a test is sometimes also mentioned.
There are many different types of validity, including: content validity, face validity,criterion-related validity (or predictive validity), construct validity, factorial validity, concurrent validity,convergent validity and divergent (or discriminant validity).
a. Content validity pertains to the degree to which the instrument fully assesses or measures the construct of interest. For example, say we are interested in evaluating employees’ attitudes toward a training program within an organization. We would want to ensure that our questions fully represent the domain of attitudes toward the training program. The development of a content valid instrument is typically achieved by a rational analysis of the instrument by raters (ideally 3 to 5) familiar with the construct of interest.Specifically, raters will review all of the items for readability, clarity and comprehensiveness and come to some level of agreement as to which items should be included in the final instrument.
b. Face validity is a component of content validity and is established when an individual reviewing the instrument concludes that it measures the characteristic or trait of interest. For instance, if a quiz in this class comprised items that asked questions pertaining to research methods you would most likely conclude that it was face valid. In short, it looks as if it is indeed measuring what it is designed to
measure.
c. Criterion-related validity is assessed when one is interested in determining the relationship of scores on a test to a specific criterion. An example is that scores on an admissions test for graduate school should be related to relevant criteria such as grade point average or completion of the program. Conversely, an instrument that measured your hat size would most assuredly demonstrate very poor criterion-related validity with respect to success in graduate school.
d. Construct validity is the degree to which an instrument measures the trait or theoretical construct that it is intended to measure. For example, if one were to develop an instrument to measure intelligence that does indeed measure IQ, than this test is construct valid. Construct validity is very much an ongoing process as one refines a theory, if necessary, in order to make predictions about test scores in various settings and situations.
e. Concurrent validity is a statistical method using correlation, rather than a logical method. Examinees who are known to be either masters or non-masters on the content measured by the test are identified, and the test is administered to them under realistic exam conditions. Once the tests have been scored, the relationship is estimated between the examinees’ known status as either masters or non-masters and their classification as masters or non-masters (i.e., pass or fail) based on the test.
f. predictive validity. This approach is similar to concurrent validity, in that it measures the relationship between examinees' performances on the test and their actual status as masters or non-masters. However, with predictive validity, it is the relationship of test scores to an examinee's future performance as a master or non-master that is estimated. In other words, predictive
validity considers the question, "How well does the test predict examinees' future status
as masters or non-masters?"
to be continued
