5 - Best practices with Rasch Analysis

Here we will discuss some general principles about how to implement the Rasch Model for your data.

• It is easiest to run the model with a large sample and few items. Samples of less than 200 people can be problematic, especially if there are a large number of items (20+) and responses to the items are not distributed evenly.
• In general, minimal data adjustment is best. Always try to see if you can make as few adjustments to your data as possible. This means the outcome of your analysis will better support your original survey instrument.
• Try to make testlets only among items that are conceptually similar. It is easier to justify combining items that are very similar (for instance, “feeling depressed” and “feeling anxious”) than items that are extremely different (for instance, “walking 100m” and “remembering important things”). If you have high correlation among items that are very conceptually different, this may point to other problems with the survey instrument that should likely be addressed.
• When recoding, try to collapse only adjacent thresholds For instance, if you see that thresholds are disordered in the pattern 2, 1, 3, 4, it is natural to try to collapse thresholds 2 and 1 because they are adjacent. It would not make sense to collapse thresholds 2 and 4 because they are not adjacent.
• When recoding, leave the first response option alone and do not recode it. This first response option represents an answer in the MDS of “no problems” or “no difficulty”, and it is best to leave this response option alone as a baseline. There is a larger conceptual difference between having “no problems” and “few problems” than there is between having “few problems” and “some problems”, so it makes less sense to collapse the first two response options than it does to collapse the 2nd and 3rd response options.
• It is normal to have to run many iterations of the model in order to find a solution that works best, especially if you have many items in your instrument.