The MBTI® instrument has come under lots of criticism recently in the popular press. For those of us who have trained people to use this instrument and have found it useful, this can be painful. We realize that the criticism often comes from wanting the instrument to do things it isn’t designed to do. And usually it is a result of not following best practices such as the ones Jane Kise has described very thoroughly in a recent blog.

I’ve been teaching people to use Psychological Type, Temperament and other models of individual differences since 1976 and conducted MBTI® Certification Programs for 17 years. At first we just took the instrument results as the indication of one’s type pattern. Then we learned that this often wasn’t as accurate as we thought it was. This inaccuracy is in part the basis for many of the critiques. The critics say they take the ‘test’ one time and it comes out different another time. The MBTI® Manual contains data that indicates that the agreement with instrument results ranges from about 60% -80% of the people agreeing with all four preferences ‘measured’ by the instrument. This really isn’t bad since as the Manual points out, chance would predict likelihood of this fit to be 6.25%. But the critiques don’t understand this. And in a way they are right. To base important life decisions such as career choice on something with an error rate that big is really not a good thing.

There is room for good critique. One of the main problems is that the proponents as well as the critics are confusing the MBTI® instrument with typologies in general. The critics may fail to realize that their resistance may be more to down side of using typologies than to the instrument. They seem to ignore that there is a downside to trait models as well even though there is a lot of data supporting them. As Jennifer Selby Long points out, the trait models also have down sides because they present certain qualities as negative and ‘less than’ rather than taking a neutral stance. With type theory these same qualities are shown to be positive aspects of one’s personality. In the end, people want to be seen in a positive light so no wonder the lay public likes type. In my view, I think there are several polarities that we can address that could move the discussion forward help us reap the rewards of good critique. It wasn’t until I read the second blog post by Adam Grant that some of the polarities became clearer. In the end, I have to agree with him that we might benefit from resolving our scientific disputes through crictical experiments with a mediator. It would be challenging, but worthwhile.

In truth personality instruments do work, but not all by themselves and not for some of the purposes the critics want them to. So here are some points to consider that apply not only to the MBTI® instrument, but to all typology instruments.

Objective vs subjective realities. I think that the critics’ mindsets come from a perspective that the only reality is objective, ‘measurable’ reality. Yet there is ample support for the influence of our mental models on our behavior. These live in our subjective realities. In my experience most of the proponents of using type models in their growth and change work seem to be insisting that these other realities are more important so they don’t put much stock in research that is objective in nature. Both the objective data gathering approach and the attention to subjective realities are important. And if we don’t realize this, energies and resources will be spent in arguments that can’t be won.

Instruments are not assessments. Before I discuss the idea of different mindsets, I want to point out a distinction between assessment and instrumentation. An instrument is just the questions or the tasks the individual is asked to do that results in a score or output of some kind. An assessment, historically, involves clinical judgment that factors in many more data points along with the instrument results. The MBTI® instrument is just that—an instrument. For typology instruments to be useful, they need to be used as part of an assessment process than takes into account many variables as well as part of an ‘application’ area. There are so many things that can make instrument results inaccurate, and therefore invalid and unreliable. The mere complexity of what it means to be human includes variables such as the mind set of the individual, how the instrument is framed to them, the proposed uses of it such as on the job where there is fear of bias versus with a coach where there is safety, the context such as work or individual interest, and it goes on and on.

Types are patterns that are stable throughout our lifetime AND we are dynamic so detecting these patterns is not easy. In the Berens CORE™ Approach we talk about how we have a Core Self that is there from birth and yet at the same time we have a Contextual Self that is very adaptable and a Developed Self that reflects the influences of both nature and nurture. So which Self is responding to an instrument? How do we ensure that personality type instruments are used in a way that helps us get at the Constant Core, yet makes space for the other selves?

An Integral Approach to Assessment. Last May, I participated in a webinar by Dr. Zachary Stein in which he presented an Integral view of assessment. Dr. Stein described an assessment method that can be applied to personality type as we take into account the many variables that greatly influence the accuracy of identifying one’s typology. Using the AQAL Matrix from the work of Ken Wilber, he described some aspects we need to attend to in our assessment methods. This matrix describes the various domains of what it means to be human. It can provide us with a map to understand some of the polarities in the arguments and perhaps help us move to even better approaches to understanding differences.

The top two quadrants apply to the individual and the bottom two apply to the collective, or groups. The left two quadrants apply to our internal subjective sense of self and relationships, and the right two apply to the external, objective aspects of being human. The matrix below shows a very abbreviated view of some of the kinds of things that show up in each quadrant.

Dr. Stein pointed out that traditional measurement ignores the upper left and the lower left quadrants. He described aspects of assessment that go with each quadrant. I’ve added my comments relative to personality type assessment using tools like the MBTI® instrument.

Upper Left: This is subjective and therefore variable with different individuals. It has to do with how the client interprets and makes meaning of the instrument questions. How does the instrument affect the client? What are his or her feelings about the situation; hopes, dreams, or fears about how the information will be used; and so on? In addition to some of these individual and subjective aspects of the instrument taking situations, we need to realize that when people take a type instrument, they are answering questions about how they subjectively value different aspects. We don’t know their frames of reference or mind sets in response to the items and the instrument-taking situation. This introduces so many variables that are not taken into account with such wide variability in frame setting from none to extensive. These same variables are at play with trait based instruments, but since results are presented quantitatively, the consequences are not so noticeable.

Upper Right: This is objective behavior and the domain of validity and reliability. Can we have confidence in the scores? Are the responses to similar items consistent? What are the scores? Most personality instruments are judged on this basis. However, they ignore that the instrument results are not actual behaviors, but self- reports of preferences, which are really Upper Left and therefore subjective. To expect a direct correlation between behavior and self-report is unrealistic. There are many influences on our behaviors, including adaptation and development. The critics ignore these other influences and expect more than the instrument is designed to deliver. No wonder the instruments aren’t working the way they are expected to in doing research or just plain test-retest for the individuals.

Lower Left: This is where the influence of cultural meanings and values comes into play and these influences greatly influence how the individual responds. There is also the issue of how the results will be used in the culture of the organization. If certain type preferences are given privilege in that culture, they will likely be seen as less desirable than others and maybe it is in the best interest of the individual to not respond from her or his core self. Relationship and cultural variables are major sources of reliability and validity variables. And the critics ignore that best practices with type models and instrument control for this. With good typology instruments and descriptions there is a lot of effort put into developing the items and descriptors to counter the effects of social desirability.

Lower Right: This is the influence of the fit between the instrument and its function. How useful will it be for the applications areas? In the arena of personality types, this becomes one of the biggest issues because while type indicates some likely talents, it doesn’t predict skill so to use it for selection or promotion is not appropriate. Yet the critics don’t get this point.

So you see, personality instruments are difficult to construct and administer in such a way that all of the areas are covered. As Dr. Stein points out, Integral psychometrics are ones that do not violate the integrity of the individuals, they adhere to the objective qualities of good instrument construction, they are used ethically, and they are shown to be useful in the systems they are being used in. Single instruments do not do the whole job. Assessment methods based on sentence completion might hold some hope, but this is expensive.

No wonder instruments don’t work…by themselves. As noted, one of the reasons personality instruments don’t work is that we are expecting an instrument to do the job of assessment. Assessment includes a much broader look than just the data points from a a single instrument. In the world of tests and measures, it includes clinical judgment. The instruments have to be part of an assessment process, not just isolated as ‘tests.’ And yet in most environments and cultures, we look to ‘tests’ to give us answers. Often the professional capable of doing an assessment is either not present and a ‘test’ is used from the web. In the world of type, the individual being assessed as also qualified to provide input into the clinical judgment. However, unless there are full narrative reports provided and encouragement to explore, this is not enough clinical judgment to make it a good idea to not have a professional involved. And even when a professional is engaged, organizations reduce the time they are given to do their job well. In addition many instrument certifications don’t provide a method to help the professional do the assessment aspect and may ignore the contracting skills needed to get the time and frame setting that will ensure a job well done. Then we wonder why the instruments come under attack!

It is up to the practitioner to get the knowledge base, skills, and tools to facilitate the assessment process. For typology, this is not likely to be the focus of brief, publisher-run certification programs, although ethics are covered. Some of the best sources of getting the knowledge base comes through organizations like the Association for Psychological Type International through their publications, training, and ethics statement.

It has been my life work to write material and create products and training that furthers this purpose, so I hope you explore the resources on this site. The Berens CORE™ Approach is an Integral approach to personality assessment and covers all the quadrants of the Integral Matrix. It provides tools, processes, and materials to engage clients in a self-discovery process that helps round out the assessment process. Check it out!

Click here to read more about the evolution of psychological type and what is on the Leading Edge as well as more about the CORE Approach. I hope you found this blog post helpful.