Scientific papers can be almost unreadable to the layman and many of the subtle clues that tell you how good a paper is are lost. For example what are the relative merits of a ABAB design over an ABBA design? But all is not lost. The British Medical Journal has published a series of articles on how to read a paper. The articles are focused on the doctors and medical papers but much of what they say applies to psychology studies as well.
"Most comparative clinical trials include either a table or a paragraph in the text showing the baseline characteristics of the groups being studied. Such a table should show that the intervention and control groups are similar in terms of age and sex distribution and key prognostic variables (such as the average size of a cancerous lump). Important differences in these characteristics, even if due to chance, can pose a challenge to your interpretation of results. In this situation, adjustments can be made to allow for these differences and hence strengthen the argument"