Introduction

The following assignment is an integration assignment. It challenges you to think critically about the course concepts and to practically engage your knowledge.

Evaluation of a Mock Test Manual

When a test administrator wishes to identify whether an instrument is appropriate for a targeted purpose/population several steps must be taken. The test administrator must familiarize him or herself with the test manual, evaluating the foundation upon which the instrument stands. This includes an evaluation of the strengths and weakness of the instrument, per the empirical evidence presented by the test publisher. For this assignment you will play the role of a test administrator evaluating the “advertised” applicability of a mock instrument called the Nelson Critical Thinking Test (NCTT). You will analyze the test properties presented in the supplied mock test manual in order to make decisions on the appropriate use of the NCTT.

Directions

Read the Abbreviated Test Manual for the Nelson Critical Thinking Test (NCTT).

After you have read the mock test manual, evaluate the test’s properties by answering the questions presented below.  Your responses should include both a definition of the measurement concept reflected in the question (e.g., reliability, criterion-related validity) and reference to specific pieces of information contained in the manual. (Hint: Clearly identify from which study information is extracted -- Study 1, Study 2, etc. -- when applicable.)

1.    What evidence is presented regarding the reliability of the NCTT?  Evaluate the meaning and adequacy of this evidence.

2.    What evidence is presented regarding the content-related validity of the NCTT?  Evaluate both the meaningand the adequacy of content-related evidence for the validity of the NCTT.

3.    What evidence is presented regarding the criterion-related validity of the NCTT?  Evaluate the meaning and adequacy of this evidence.

4.    What evidence is presented regarding the construct-related validity of the NCTT?  Evaluate the meaning and adequacy of this evidence.

5.    Describe one use of the NCTT that you believe is justified, given the information provided in the manual.

6.    Describe one use of the NCTT that you believe is not justified, given the information provided in the manual.  In order for the NCTT to be used in this way (or for this purpose), what information would you like to see in the manual?

Helpful Hints

Before completing the assignment, take these steps to organize your thoughts.  (These steps are for your use only.  Do notsubmit for grading purposes.)

1.    Before reading the mock test manual, make a quick reference list, defining the relevant terms from the question set.

2.    Read the Abbreviated Test Manual for the Nelson Critical Thinking Test (NCTT).

3.    Create a table similar to that presented below. A table will help you organize and evaluate the information presented in the mock test manual, deciding whether it provides you information on reliability or validity. For example, if you are looking at validity evidence from a study -- decide what type of validity the test results provide and annotate it in your table. Once you have done this you will be ready to tackle the questions for the assignment.

 

Study/Event	Reliability	Validity	Meaning	Adequacy
Study 1	X		(What does the study tell you?)
Study 2				Your notes here.
Study 3		X(name the type of validity evidence presented)
Study 4
Study 5
Study 6
Test Development/Item selection

 

 

Attachments:

• 1501168430-hw assignment 3.docx