Posts

Experimental Methodology in NLP Research

/in /by

Introduction

Einspruch & Forman (1985) criticised 29 research studies on the basis of 6 methodological errors, concluding that a large amount of psychological research into NLP concepts was invalid because of errors and oversights in experimental design which compromised its scientific credibility. Sharpley (1987) rightly questioned the validity of some of these methodological objection on the grounds that they unnecessarily discounted a large number of potentially valuable results. He maintained that even when factoring in methodological errors, significant results had been attained by the research.

However, an analysis of the literature reviews and the studies to which they refer does reveal some consistent oversights of certain distinctions that are vital for scientific enquiry into NLP to succeed. While these distinctions are already available to well trained NLP graduates (given that they are core patterns presupposed in many applications of NLP), they will be explained in detail here because of their frequent occurrence in experimental research into NLP. The importance of integrating this knowledge into future design methodologies in NLP research cannot be overemphasised.

The most important methodological issues raised by Einspruch & Forman can be grouped into three patterns, all based on the associative nature of the human nervous system. This basic pattern of human functioning provides important insights into the significance of context in experimental studies and in theoretical considerations.

Pattern #1: Humans Are Influenced by Associations

Psychological Underpinnings

The basic psychological understanding of association was famously illustrated by Pavlov (1927) in his classic experiments with ringing bells and salivating dogs, where he trained the dogs by consistently preceded their feeding time with auditory stimuli such as bells and tuning forks. After a training period he then reproduced the stimuli in the absence of the food and found that in itself, the auditory stimuli was enough to make the dogs salivate by virtue of the learned association. Associative learning has been explored in detail by thousands of psychological researchers, but the lessons learned from our in-depth explorations have occasionally been overlooked.

NLP Research

Einspruch & Forman draw attention to one example in particular. Dorn (1983) attempted to determine participants’ PRS by using three different methods of assessment. One method involved participants selecting their preferred predicate from each of 18 sets of three words; one visual, one auditory, one kinaesthetic; assuming that selecting one out of three words would be done on the basis of a preferred representational system, as opposed to having its own specific associations which influence choice of one word over others in the triad.

Pattern #2: Humans Function Within, and Are Influenced by, Context

Psychological Underpinnings

This pattern corresponds to basic experimental design considerations. It has been a long-understood concept that similarity between the test context and acquisition context is an important factor in memory experiments. It is also well known that “elements of the training context (i.e., background cues) may act as conditioned stimuli during a test trial” (Miller & Schachtman, 1985). These observations draw on associative processes such as classical conditioning (e.g. Rescorla & Wagner, 1972). Even at the time of Pavlov and his contemporaries, it was widely recognised and understood that, “despite the experimenter’s best efforts to make the subject attend exclusively to the nominal controlling stimuli, the test context [influences] behaviour through direct associations between it and any reinforcers that [have] previously been presented” (Miller & Schachtman, 1985).

What is clear in the psychological literature is that the confounding and influential effects of context significantly affect processing across the spectrum of human cognition; over such wide ranging topics as learning, memory and recall, language interpretation, problem solving and perception (see Balsam & Tomie, 1985, for detailed reviews of the impact of context in these areas of cognition).

The impact of context is illustrated by a classic example of context-dependent memory; that the processing of memories is heavily influenced by the context within which learning and recall take place. Baddeley and Godden (1975) tested the memory of participants in two different environments: underwater and on land. When words were recalled on land, participants recalled correctly 37% of words learned on land, compared with 23% learned underwater. When words were recalled underwater. participants recalled 24% of words learned on land compared with 32% learned underwater. Endel Tulving (considered by many to be the father of learning and memory experimentation in psychology), in 1983, formalised this very idea with his well-known theory of encoding specificity in learning and recall.

NLP Research

With regard to the research into NLP, Einspruch & Forman (1985) noted correctly that “the representational system in which information will be stored or from which it will be retrieved is highly contextualized (i.e., varies with the situation), and this context will directly influence the system used.” These researchers were particularly perceptive in noting that “context plays an important role in determining the meaning as well as the structure of any communication.” This is well-known in linguistics and psycholinguistics (e.g. the involvement of context in the resolution of syntactic sentence ambiguity, Mitchell, 1994).

Einspruch & Forman criticised the experimental results of Gumm, Walker, & Day (1982) for neglecting to control for context. After interviewing experimental participants to determine their PRS, they were moved to a room surrounded by curtains, where their heads were placed in a restraining device so that eye-movements could be filmed. This severe contextual alteration would certainly be sufficient to abolish any effects of from dominant representational system usage.

“The processing of memories is heavily influenced by the context within which learning and recall take place.”

Gumm et. al. attempted to determine the PRS using 3 different techniques: predicate tracking, eye movement monitoring and self-report. Their finding that “each assessment method was shown to be biased toward revealing a particular representational modality,” and that such a bias “may be the result of the counsellor’s primary employment of a particular method of assessing the client’s PRS,” illustrates a recognition for the influence of context and their lack of control for it in their experiment. People inevitably adjust processing strategies according to both explicit and implicit demands of a presenting task, and such experiments only serve to illustrate this point further.

Pattern #3: Language Creates a Context Within Which People Respond

Current Understanding

The current representational system model proposes that the system a person accesses is heavily influenced by:

  • the current context and the type of question asked (which together create task demands);
  • the way a person represents the particular context which is being asked about.

NLP Research

Gumm et. al. measured PRS by “recording the position of the initial eye movement following the end of each question.” An understanding of eye-accessing strategies precludes this method of accessing some type of a stable representational system.

Bandler & Grinder (1979), when discussing eye-accessing cues, draw attention to the possibility that when you ask someone what their mother looks like, they may first access the auditory system (there are many reasons this may happen – a person may talk to their mother more often than they see her, etc.), subsequently check their feelings (that may give them a response which indicates that they are indeed listening to their mother), then access an image of her from memory. Thus there may be, on occasion, a sequence of accessing cues (called a strategy) which take place, which will be different depending upon task demands, type of language used to elicit the strategy and habitual responses, among others.

Gumm et. al. also used a self-report measure, asking participants what they thought their primary representational system was. The results of such investigations are likely to depend in large part on the verbal frame presented by the experimenter. For example, Elich, Thompson, & Miller (1985) told subjects that “personality characteristics would be assessed through the use of imagery.” Subsequently, “subjects were asked to describe the image or sequence of images evoked by the question,” and “subjects were asked to imagine and describe their favourite experience in order to assess spoken predicates.”

The unsurprising results of this study were that:

  • “Imaging did not occur exclusively in the single modalities suggested by Bandler and Grinder but involved the multi-modal experience of a visual image followed by the image intended by the question.”
  • “With the auditory and kinaesthetic questions, the most common occurrence was a visual image followed by either the auditory or kinesthetic image.”
  • “The images evoked by the control questions were visual.”
  • “Most subjects regarded themselves as visualizers.”
  • “Most predicates were visual.”

Elich et. al. recognised that “the term imagery may have set an expectation to have visual images and use visual predicates. If so, PRS is heavily influenced by situational variables like language.”

Falzett (1981) had participants read and generate an internal response to 6 questions in order to determine the person’s PRS. Unfortunately the content of these questions are leading enough in terms of sensory predicates to be good candidates for eye-accessing cue elicitation questions.

Two questions were kinaesthetically biased:

  • I’d like you to think about the last time you were really comfortable.
  • What was the last thing you touched that you really enjoyed?

One question was auditorily biased:

  • What was the last song you heard before coming here?

And only three of the six were adequately general to examine strategic preferences without leading in any way:

  • I’d like you to think of a time when you had accomplished an important goal.
  • What is the last thing you remember before you came in here?
  • I’d like you to now think of a pleasant childhood experience.

The following elicitation requests were designed to elicit verbal responses about which confederates could feed back predicates to gain rapport. However, they are structured in such a way that makes them likely to elicit belief strategies (to find out how a person knows something to be true) and hence create task demands which differ from those desired in the experiment (i.e. eliciting PRS predicates):

  • When you knew someone understood you
  • When someone loved or cared about you
  • When you knew someone trusted you

Out of 24 participants, Falzett found “only 3 who were not predominantly kinaesthetic” in their responses to these questions. This may simply indicate that most people in the study had a tendency to use a kinaesthetic component as a significant part of their belief strategies.

Examples of more general experiences which would control for context and potential mental strategies would be:

  • A boring experience in the same context
  • A common experience in the same context

These would be less likely to have leading or biasing factors which confound results.

Additionally, Falzett’s finding that matching predicates increased trustworthiness may simply be an artefact of confederate’s usage of predominantly kinaesthetic predicates, which may have imparted the confederate with an air of genuine self-expression and thus, trustworthiness. Without control sets, however, such a hypothesis is impossible to verify.

The observation by Falzett that eye-accessing cues yielded the best results for determining PRS was dangerously generalised by Dorn (1983) to mean that eye-accessing cues are “most conducive to research on NLP and should be employed over the predicate usage method.” This consideration was likely made because of the inconsistencies of research findings surrounding the PRS. However, methodological errors like those above creates a sense of chasing one’s own tail in terms of attempting to make conclusions about experimental findings in NLP research.

Summary

While Sharpley (1987) criticised Einspruch & Forman (1985) for dismissing numerous NLP research papers on the basis of their unfamiliarity with and lack of training in NLP, it is important to recognise that good quality training can avoid certain methodological pitfalls which would have been apparent to those who had undergone adequate training.

However, the ability to understand the issues uncovered above is by no means confined to NLP practitioners, given that they are all well known psychological effects and widely accepted in the psychological research community as being important considerations to take into account during the methodological design phase.

Thus, while psychologists and cognitive scientists undergoing research into NLP concepts do not necessarily have to have undergone NLP training, the additional filters and perspectives of comprehensive NLP training can allow a more coherent explanation of experimental results in terms of the patterns of behaviours occurring within experimental contexts

Given that even the research which had found effects supporting NLP concepts suffered from various methodological confounds, it is not easy to make any generalisations about the validity or use of past research into NLP. What is clear, is the importance of careful, well planned research into NLP in the future, to assist the development of the field of NLP as a whole.

“Careful, well planned research into NLP in the future is important to assist the development of the field.”

References

Balsam, P.D. & Tomie, A. (1985) Context and Learning. Hillsdale, NJ: Lawrence Erlbaum Associates.

Bandler, R., & Grinder, J. (1979). Frogs into princes : neuro linguistic programming. Moab, Utah: Real People Press.

Dorn, F. (1983). Assessing primary representational system (PRS) preference for Neuro-Linguistic Programming (NLP) using three methods. Counselor Education and Supervision Vol 23(2) Dec 1983, 149-156, 23, 149-156.

Einspruch, E. L., & Forman, B. D. (1985). Observations Concerning Research Literature on Neuro-Linguistic Programming. Journal of Counseling Psychology, 32(4), 589-596.

Elich, M., Thompson, R. W., & Miller, L. (1985). Mental imagery as revealed by eye movements and spoken predicates: A test of neurolinguistic programming. Journal of Counseling Psychology, 32(4), 622-625.

Falzett, W. (1981). Matched versus unmatched primary representational systems and their relationship to perceived trustworthiness in a counseling analog. Journal of Counseling Psychology, 28, 305-308.

Godden, D. R., & Baddeley, A. D. (1975). Context-dependent memory in two natural environments: On land and under water. British Journal of Psychology, 66, 325 – 331.

Gumm, W., Walker, M., & Day, H. (1982). Neurolinguistic programming: Method or myth? Journal of Counseling Psychology, 29, 327-330.

Miller, R.R. & Schachtman, T.R. (1985): The Several Roles of Context at the Time of Retrieval. In P.D. Balsam & A. Tomie (Eds.), Context and Learning. Hillsdale, NJ: Lawrence Erlbaum Associates.

Pavlov, I. P. (1927). Conditioned reflexes. London: Routledge and Kegan Paul.

Mitchell, D.C. (1994): Sentence parsing, in Morton Ann Gernsbacher (ed.), Handbook of psycholinguistics, Academic Press

About the Author

Richard Thompson, BSc. (Cognitive Science), is a Graduate of Exeter University, and is a freelance writer and web consultant. He holds the Graduate Certificate in NLP, from INSPIRITIVE, and enjoys receiving responses to his work.

Article content copyright 2006. Richard Thompson. All rights reserved.

All other material copyright 2006. INSPIRITIVE Pty Ltd. All rights reserved

If you found this article useful hit the share button!

[polldaddy rating=”7958655″]

NLP Research Literature Overview

/in /by

A Summary of Research Reviews

A large amount of research on NLP was done in the early 1980s on a construct called the “Preferred Representational System” (PRS).

It has been surmised that proponents of NLP have, in the past (Sharpley, 1987) had “little to support them and much to answer to in the research literature.” Commenting on the research which had already been done by that time, Sharpley (1987) commented to researchers that “there [was] little use to the field of counseling research in further replications of previous studies,” until researchers had taken the time to perform a “careful meta-analysis of the large amount of data already gathered.”

Here it is hoped that such an undertaking will be able to promote, as Sharpley suggested, “future research that can contribute new data … via methodological advances [and a] consideration of different aspects of NLP.”

The majority of research into NLP concepts was primarily concerned with the concept of a “Preferred Representational System,” or PRS. It is important to understand the issues surrounding this concept before examining the research.

A large number of these studies only have limited value as evidence, however, due to significant confounding methodological and theoretical issues.

One common misunderstanding of the field of NLP in the psychological research is the important idea that NLP is an epistemology and a methodology, not a single theory or model.

“NLP is an epistemology and a methodology, not a single theory or model.”

Once these issues are understood, it will be possible to engage in useful and interesting research into the field of NLP, through adhering to a number of factors which are important in designing good research into NLP.

Sharpley’s (1984) Review: “NLP Is Unsupported in The Literature”

The first review of the experimental literature in NLP by Sharpley (1984) suggested little supportive evidence and a large amount of data opposing the validity of the concept of a PRS.

The review cited 15 studies which were concerned with determining:
  • The presence of a PRS,
  • Adequate methods to discover a PRS,
  • The effects of matching PRS with verbal predicates

However, the review did not consider numerous methodological dimensions which are important in examining and evaluating NLP research.

Einspruch & Forman’s (1985) Critique of Methodological Problems

Einspruch & Forman (1985), in a later review of 39 empirical studies (including all of the 15 initial studies) promoted the idea that NLP research is, in theory, testable and verifiable, but that past research was fraught with methodological confounds such as:

  • A lack of understanding important psychological patterns such as the effect of context;
  • An unfamiliarity with NLP as an approach to therapy;
  • Lack of familiarity with the “meta-model”;
  • Failure to consider the role of stimulus-response associations;
  • Inadequate interviewer training in rapport-building;
  • Logical mistakes

As a result of these criticisms, Einspruch & Forman (1985) classified all 39 studies as unreliable and concluded that it was “not possible at this time to determine the validity of either NLP concepts or whether NLP-based therapeutic procedures are effective for achieving therapeutic outcomes,” and that “only when well-designed empirical investigations are carried out may we be assured of NLP’s validity as a model of therapy.”

Bearing in mind New Code principles, specifically the significance of framing and context, at least two of these can be considered useful guiding principles in evaluating past research and planning future research.

1. Lack of understanding patterns of behaviour and control for confounding contextual factors surrounding the application of NLP models

Einspruch & Forman (1985) recognised that people behave in a way that is dependent upon the environment – the context – within which they find themselves. 15 studies were criticised for not taking this into account. An additional pattern not incorporated into studies was that of individual calibration; people are systematic with their own behaviour but it is potentially erroneous to assume that behaviour is systematic across any given population of people, and thus, procedures and interventions should be applied at the individual level and calibrated to each person’s characteristics.

2. Failure to consider the role of stimulus-response associations

Einspruch & Forman (1985) also understood that the basic concept of association was a powerful influencer of behaviour, and that people were likely to respond based on the associations they had learned, if the experimental settings did not control for this possibility.

These, and other examples of methodological errors in past research, can be summed up by using the following dimensions, which are core concepts to NLP as a discipline, and well recognised in psychological literature:

  • Associations influence processing
  • Features of a context (contextual markers) function as influential associations
  • Language creates task demands , serving as a contextual marker which provides a frame, thereby influencing responses

It is important to understand these concepts in order to conduct useful research.

Einspruch & Forman (1985) made other important and often misunderstood observations:

  • Representational systems are an important part of NLP, but are only one model within NLP
  • Researchers should be adequately trained in NLP so that the procedures and interventions generated can be used within the presuppositions contained in the model

Sharpley’s (1987) Reply

Although Einspruch & Forman did develop some useful methodological distinctions, Sharpley (1987) objected to the resultant dismissal of valuable research and produced a doubly-binding suggestion that either:

  • The past experimental results a lack of conclusive effects, or that
  • The procedures examined were not able to be adequately assessed.

Sharpley (1987) considered that not all of the criticisms Einspruch & Forman (1985) raised were reasonable, and resulted in a somewhat unnecessary dismissal of research.

For example, the criticism that experimenters had a “lack of familiarity with the meta-model” because they measure nominalisations such as anxiety, ease, empathy and hostility (used to discount the validity of 3 studies), can be upheld only insofar as the particular psychometric inventories used can be questioned as reliable.

Einspruch & Forman rejected the validity of 7 studies for an “unfamiliarity with NLP as an approach to therapy,” based on the criticism that asking questions about a client’s past displayed a misunderstanding of the nature of NLP as a generative as opposed to an archaeological approach. This appears to be a rather extreme view and is not necessarily true, in that theoretically, basing experiments around questions in a client’s past is not enough in itself to invalidate the testing of NLP.

Sharpley (1987) critisizes their exclusion of 12 studies because “researchers were not adequately trained in NLP” by commenting that “it is difficult to accept that none of the studies were performed by persons with enough of an understanding to perform the various procedures that were evaluated.” Unfortunately this is difficult to confirm or deny, given that up until now, there has been no officially recognised standard for NLP practitioners.

Sharpley (1987) cited a further 7 studies which showed no evidence in support of the concept of a PRS. However, these studies also contained various methodological issues. Elich, Thompson, & Miller (1985) and Graunke & Roberts (1985), for example, are among those studies discussed as examples of flawed methodological design.

Sharpley’s Conclusions

Some of Sharpley’s (1987) conclusions (which mirror Elich, Thompson, & Miller’s, 1985), provide similar conclusions to the current model of representational systems:

(a) PRS may change over time,

(b) It is not certain that PRS exists,

(c) PRS may merely reflect current language style,

(d) PRS may be heavily influenced by language.

While these conclusions are similar to our current position on representational systems – which mirror John Grinder’s – the methodology used to arrive at them leaves a great deal to be desired and points directly to factors to consider in any analysis of current research, and during the design phase of future research projects.

NLP as a Field and a Discipline

Unfortunately Sharpley and others have made a common mistake, revealing a failure to recognise that NLP is a methodology for creating outcome-oriented models, in considering that the PRS constitutes one of the basic tenets of NLP.

PRS is one aspect of a single model of NLP, and as such it is a mistake to assume that lack of evidence for one model within the field of NLP in any way compromises the validity or utility of subsequent NLP research, or on the field of NLP as practised currently.

(Note: If you would like to learn more about the NLP as a field you can, claim your free copy of , our e-book ‘Neuro-Linguistic Programming; An Overview of the Field’, by Chris Collingwood, NLP Trainer. For a limited time only, here).

Sharpley concludes by putting NLP “in the same category as psychoanalysis, that is, with principles not easily demonstrated in laboratory settings but, nevertheless, strongly supported by clinicians in the field.”

Our proposal is that NLP can and should be researched, quantitatively and qualitatively.

Requirements For Future Research

If adequate research is to take place, concise, highly specified and empirically verifiable descriptions of the models of NLP need to be in place.

In order for this to happen there must be an agreement upon the definitions within the NLP community.

The Graduate Certificate NLP is the first step towards establishing an accepted standard, and along with Grinder & Bostic-St.Clair, Inspiritive and its collaborators are moving towards developing agreed-upon vocabulary and explicit definitions of the patterns and models of NLP.

If you would like to take part in this project, please contact Inspiritive. This is a vital endeavour if the field of NLP is to reclaim its roots within the Cognitive Sciences and consolidate its establishment as a valid field of endeavour, study and research.

References

Bostic St. Clair, C., & Grinder, J. (2001). Whispering In The Wind. Scotts Valley, California 950666: J & C Enterprises.

Einspruch, E. L., & Forman, B. D. (1985). Observations Concerning Research Literature on Neuro-Linguistic Programming. Journal of Counseling Psychology, 32(4), 589-596.

Elich, M., Thompson, R. W., & Miller, L. (1985). Mental imagery as revealed by eye movements and spoken predicates: A test of neurolinguistic programming. Journal of Counseling Psychology, 32(4), 622-625.

Graunke, B., & Roberts, T. (1985). Neurolinguistic programming: The impact of imagery tasks on sensory predicate usage. Journal of Counseling Psychology, 32(4), 525-530.

Sharpley, C. F. (1984). Predicate matching in NLP: A review of research on the preferred representational system. Journal of Counseling Psychology, 31(2), 238-248.

Sharpley, C. F. (1987). Research findings on neurolinguistic programming: nonsupportive data or an untestable theory? Journal of Counselling Psychology, 34, 103-107.

About the Author

Richard Thompson, BSc. (Cognitive Science), is a Graduate of Exeter University, and is a freelance writer and web consultant. He holds the Graduate Certificate in NLP and enjoys receiving responses to his work.

Article content copyright 2006. Richard Thompson. All rights reserved.

All other material copyright 2006. INSPIRITIVE Pty Ltd. All rights reserved

If you found this article useful hit the share button!