Response to Bandura
艾尔
Table 1 from Bandura, 2012
Methodological Deficiencies in Tests of Perceptual Control Theory for Negative Self-Efficacy Effects
Vancouver, Thompson, & Williams (2001)
- Deficient assessment of self-efficacy
- Guessing game with random sequential disconnectedness
- Participants discarded because they guessed the correct solution on first trial
- Posited key mediator, "perceived discrepancy," never measured
Vancouver, Thompson, Tischner, & Putka (2002)
- Deficient assessment of self-efficacy
- Guessing game with random sequential disconnectedness
- Deletion of goal comparator rendered control theory untestable
Vancouver, More, & Yoder (2008)
- Deficient assessment of self-efficacy
- Performance tasks with random sequential disconnectedness
- Pseudo "do your best" goal substituted for an explicit goal comparator in test of control theory
- Discrepancy not judgable against an indefinite "do your best" goal comparator
Vancouver & Kendall (2006)
- Deficient assessment of self-efficacy
- Participants discarded for "illogical responses" judging themselves inefficacious for low grades
- Assessment of strength of self-efficacy abandoned
- Posited "perceived preparedness" mediator not measured
- Effort tested by retrospective self-report
- Loss of discriminative information by converting continuous exam scores to categorical grades
- Lack of variance in goals for testing performance effects
- Past performance deleted from reported hierarchical linear modeling (HLM) analysis
- Use of a dated HLM program
Other issues raised by Bandura (in press) deserve comment. These will be listed and address soon.In the meantime, below are the responses to the items in the table above.
Deficient Assessment (2001; 2002)
Th
Deficient Assessment (2008)
Bandura never specifies the measurement deficiency in the 2008 study, so I do not know what his problem was there. Return to Table 1
Deficient Assessment (2006)
英航
Random sequential disconnectedness (2001; 2002; 2008)
The next false deficiency had to do with something Bandura (in press) called random sequential discontentedness. This was a presumably a problem in the Mastermind studies (the 2001 and 2002 papers) and the 2008 study where we used a task called the hurricane game. He is not clear exactly what he means by this, but it seems there are three possibilities. First, he may be referring to his belief that the tasks have no skill component (i.e., performance is based on luck). This was one of Bandura and Locke's (2003) main criticisms of the Mastermind game, which I addressed in my 2005 response. The most obvious counter is that we had evidence that people varied in how good they were at the game, which could not happen if it were purely a game of chance. Indeed, the game is called Mastermind because it takes thought to do it well.
The second meaning might be that performances do not build on previous performances, which means we cannot generalize to cases where performances do build on previous performances. Such building on previous performances is not uncommon, but then neither is it the case that performances are independent. It is an empirical question whether the effects generalize to such situations.
Th
Discarded Participants (2001; 2002)
We did not discard any participants. We discarded a game if the solution was found on the first attempt because that would be performance based on pure luck. Indeed, this "deficiency" appears to be used as evidence of the random, guessing issue rather than some independent issue. However, as note in my 2005 response, there is an element of luck in the Mastermind game, but once an individual makes a guess there is information available that can be used to inform subsequent tries. Performance was largely based on how well one used this information, a point we demonstrated clearly in the 2002 paper. Return to Table 1
Mediators (2001; 2006)
一个
Deletion of goal comparator (2002)
Th
在
Do Your Best Goal (2008)
Wi
Deleted Participants (2006)
No participants were deleted from the study for illogical responses. But see the next issue regarding the strength measure of self-efficacy. Return to Table 1
Abandoned Strength of Self-Efficacy Measure
它
所以
Retrospective Assessment of Effort (2006)
它
Continuous to Categorical (2006)
Another true "deficiency" is that we translated continuous exam scores into categorical grades. Well, actually, that is only partly true. The exam scores were translated into 11 grade levels (letter grades with pluses and minuses) which are essentially interval scale, not categorical. What might bother Bandura is that some information was lost in that translation. However, that loss of information could only weakened our findings, not created a spurious effect. Indeed, it is knowable how much the finding was weakened. Specifically, to correct for the biased we could simply multiple the effect sizes we found by 1.012. Given that, it hardly seems worth reading this paragraph. Return to Table 1
Lack of Variance in Goals (2006)
This "deficiency" is also true. That is, we had very little variance on goals. Not sure what he expected us to do or have done about that. It is what it was. You measure your variables and report your findings. Not sure what the methodological deficiency was. Return to Table 1
Past Performance Deleted from HLM Analysis (2006)
This complaint seems to stem from an analysis Bandura had done on the Vancouver and Kendall data. Apparently, they found that past performance was significantly negatively related to subsequent performance, though I am not sure given the effect nor probably of Type II error level was reported. Assuming the above though, Bandura seems to think this is remarkable because (a) he cannot understand how it could be and (b) I always complain about the lack of past performance as a control in other people's studies. Let me begin with the second issue.
我
体育
The above still does not explain why past performance would be negatively related to subsequent performance. One possible answer is that past performance is positively affecting self-efficacy, which is negatively affecting subsequent performance. Given that self-efficacy's affect on subsequent performance is not very strong, I would not expect the indirect effect to be very strong. Bandura did not tell us what the effect was. I do not remember if I even looked at the relationship. Indeed, I has not planning to look at subsequent performance at all, but that is discussed below in the next and final issue. Return to Table 1
Dated HLM Program (2006)
The last "deficiency" Bandura mentions in his table is that we used a dated HLM program. Well, again, this is only partly true. At the time we used it, it was up-to-date. Time has passed and now it is out-of-date. So "deficiency" seems harsh. But the issue here is that Bandura claims that if one used a more up-to-date program, the negative self-efficacy with subsequent performance relationship would become non-significant. He notes that Stajkovic did that analysis for him. Well I redid it as well and they are correct. The relationship is no longer significant at the .05 level. The new p-value was .051 (the original was .046). That is, I found that the barely significant negative effect for performance is now barely non-significant when using a presumably better software program. The effect size estimate did not change.
所以
A semi-interesting aspect of this final point is that I never really expected a negative performance effect in this study. The data was likely to be too messy for what is a weak effect, as I have always argued the negative effect of self-efficacy on performance would be. Specifically, for a negative effect to occur, the students would have to be pretty badly miscalibrated and rely heavily on this miscalibrated belief to determine their study time. Because we were not trying to actively miscalibrate them (like a direct intervention might), it seemed any natural effect would be hard to detect. We did not even look at the effect of self-efficacy on subsequent performance until a reviewer asked for it. Our main purpose for conducting the study was to examine the effect on effort allocation (e.g., planned study time). That effect remained negative even using the latest software (as of this writing). Return to Table 1