Research design: Internal and external validity issues?

Question

I'm interested in whether exercise improves well-being and whether such an effect depends on sex of the participant. Using an appropriate psychometric, I will examine their well-being scores before and after a typical training session in 10 males who are recruited from a local senior rugby team and 10 females who are recruited from a local junior athletics club. What are the potential validity issues and how would I overcome them. I was thinking of using a mixed model design to analyze the data?

Answer 1

First, you really don't have a control group. Rather than test athletes before and after exercise, you should test athletes and non-athletes. Better yet, you could replace the dummy variable (athlete vs. non-athlete) with a continuous variable (such as the number of times a week the respondent execises or the amount of time the respondent spends exercising weekly).

Second, 10 is a very small group size. You need at least 30; better yet, 100.

Answer 2

Is "well-being" measurable? Frankly, in my period as a grad student fluff was not measurable. Sure you can develop a Likert to measure factors that "may" influence well being, But that is a pretty nebulous and would get a prosem professor pretty PO'd if you submitted it to him.

the 1st validity issue is what you define as your measure.. Is what you are trying to measure something that can be objectively measured and are the results repeatable. That is the rule of scientific study.

Someone mentioned having a control group that provides Content validity, but if the premise of the study is invalid to begin with then content validity has no value. And the size of your groups provides for statistical validity (variations in population.) determining the mean score of the population of ten is going to be different in a statistically valid population of 300. The great the population the less skewing you will have in your scores.