How to do a Modelling Project - Section 7
Stage 4: Testing your Model
Different forms of testing occur throughout the modelling process. The primary purpose of testing is to get feedback from:
The 'real world'
Testing your model with the exemplar
a. Test the components and steps of your model for accuracy.
As you gather information from your exemplar you can recap (in their words) as much of your model of their behaviours, abilities and states as you have. This will give them a chance to evaluate your description for accuracy.
Use your sensory acuity to calibrate that the pace of your
description enables the exemplar to 'try on' your model of them so they can compare it to their own experience, component-by-component
Every response you get from your exemplar is feedback as to
the accuracy of your model. They are the world's expert on
their model, and at this stage, that's what you are attempting
to reproduce. Anything they think is confusing, illogical, or that
doesn't fit, is a signal that your model is incomplete.
b. Test the logic of your model for accuracy
After you have confirmation of the accuracy of your
model from the exemplar, you can start to make predictions as to what
the exemplar would do in some as yet
The aim is to test if your understanding of the exemplar's logic
enables you to go beyond what you have been specifically told or
Testing your model on your own
'Try on' your model by 'running it through' your system
Can you run the model - from 'before', when the starting
Test criteria are triggered, through 'during' the
Operations, until the ending Test criteria are met, and on
to Exit 'after' (TOTE model)?
Would you expect to get the required results?
Does it all fit together?
Can you break it - under what conditions would you not get the
Are there other contexts where the process would also be applicable?
At this stage you are acquiring the model 'for the moment'.
You are not seeking to integrate it with any other models,
instead you 'put them aside' while you run your tests. In other
words, you are self-modelling to obtain feedback from your own system.
Testing the model for real
Having used your own neurology to test your model your outcome changes. You now seek to
test for the degree to which you can reproduce the required
results. You want to compare the results you get with the results your
exemplars get. To do this you need feedback from the external world.
Two ways to do this are:
a. Prepare safe 'test conditions'
Taking into account the ecology of the wider system
and depending on the potential effects of your model not working, you
may want to establish some 'test conditions' in which to test the model's
efficacy and your competency.
b. Go 'live'
The ultimate personal test. Can you get similar
results to your exemplars under similar conditions? And can you do
that consistently and under a variety of conditions? Steve Andreas said that when he constructed a new model for change, (i.e. a new NLP technique) he had to test it out with 20-30 clients before he was confident it would account for the majority of cases.
Remember, your model may work perfectly but you may not yet have
enough background knowledge or experience of running it to get the
same results as your exemplars. Acquiring Einstein's problem solving
strategy won't make you an Einstein overnight, but you can expect it
to give you access to a different way of thinking about problems and
to a wider range of solutions than you had before.
Other acquirers testing the model
If your modelling project is for other people (who were
not involved in Stages 2-4), your outcome
for testing changes again. Your design for an acquisition process
(Stage 5) should include testing by the acquirers. The feedback you
want now is: To what degree are the results the acquirers get
similar to those achieved by your exemplars.
And to reiterate: Test, get feedback, adjust model; test again, get feedback, adjust; etc. ...