In a Customer Interview What You Ask About Is Not a Judgement but the Observation of a Single Instance

About once a week, I suppose. Sentences of this shape line up in the record of an interview.

As a sentence it is an answer. The other party replied sincerely, and the interviewer appears to have heard what they wanted to hear. All the same, this sentence is not a description of behaviour. It is the result of gathering several events in the head, and in the gathering the frequency was rounded and the exceptions dropped. The three weeks with a gap in them are not inside this sentence.

The advice to ask about behaviour rather than opinion is already widely shared, and the advice itself is accurate. All the same, the question what do you do asks about a habit, and so demands the work of gathering from the other party. Changing the direction of the question alone does not change the shape of what comes back.

Three things can be resolved here. First, by what mechanism a summary comes back when you meant to ask about behaviour. Second, what to treat as the thing that comes back when you ask for a reason. Third, where the skew shows up when the people you can interview are skewed.

One question remains unadjudicated. Whether what you heard matters to your own design. Events can be captured, the record separated, the range written out, and it can still fail to bear on the design. This adjudication cannot be carried out on the side of the interview. Nor does a figure emerge for how many people or how many minutes suffice, since the degree of processing varies with the person and the behaviour, so the necessary count does not settle in advance.

The harm of the mix-up sits where the record does not show it. There is a recording, there is a transcript, several people have been interviewed. The more the form is in order, the harder it is to notice that the substance is a summary. What an orderly form guarantees is only that what was heard was correctly preserved; what happened before the preservation sits outside the form.

And a design built from summaries goes wrong quietly. It is built to match the other party’s self-image, so showing it to that same party brings agreement back. Agreement looks like evidence of having confirmed. That it fails to mesh with actual behaviour becomes clear when the response after delivery is thin, and by then the interview record has already left your hands.

How answers get processed, the unit of question at which that processing does not occur, and the range within which a skew in who can be interviewed bears on the conclusion. Three things, handled in succession. None of them turns on the interviewer’s skill; all turn on conditions in the question and the setting.

How Far Has the Method of Customer Interviews Been Organised

A customer interview is the work of hearing directly from somebody actually using your product or service. It is described as something done to learn circumstances that figures alone will not show.

The method gets shown as an order of preparation. Settle first what you want to know. Select the people. Ask and fix the date. Prepare a question sheet. On the day, warm the room before entering the subject. Obtain permission to record.

There is advice on designing the questions. Do not use forms that end in yes or no. Ask about behaviour rather than opinion. Ask how. Do not lead the answer.

Advice on the manner of asking comes attached. Do not cut in once the other party starts talking. Do not fill silences. Dig into a phrase that catches your attention on the spot. Do not move on assuming you have understood.

There is advice on analysis as well. Transcribe the recording. Pick up words that recur across people. Ask several people the same thing and look at where they overlap.

The advice lined up here is accurate as practice. Arrive without a question sheet and you miss things; ask for opinions and opinions come back. Organise from memory without a transcript and only what stayed vivid survives. None of these steps can be dropped.

One premise sits at the bottom of the set. The other party knows about themselves and will answer if asked. Ask in the right way and the right information comes out. It has been organised as a problem of extraction.

This does not become an argument that the manner of asking is unnecessary. That the form of a question changes the answer is a fact, and the design genuinely bears.

What falls outside the field of view once it is organised entirely as extraction is the processing occurring on the other party’s side. Answering is not merely the work of retrieving a memory. Retrieving, they also organise and make the account cohere. A coherent answer is easier for the interviewer to receive as well, and that ease of reception is what makes the processing hard to see.

One reading has it that what the person says is the fact. What was said is a fact. Whether what was said depicts their behaviour accurately is another matter. Without distinguishing the two, you use a processed self-image as material.

Where they are not distinguished, the shape of the later trouble is settled. The design gets built to match that self-image and goes out without meshing with actual behaviour. That it failed to mesh cannot be found by rereading the interview record, because the record side was preserved correctly.

There is a reason the list of advice does not touch the processing. What the list handles are the items the interviewer can control: the form of the question, the making of the setting, the taking of the record. Processing on the other party’s side lies outside the range the interviewer can operate, so it does not enter a list of procedures. Its absence is not an omission but the consequence of the line the list drew. What lies outside the line does not appear no matter how carefully the inside is executed.

Told to Ask About Behaviour, You Still Get a Summary

Change what do you think to what do you do and the shape of what comes back sometimes does not change.

The question what do you do asks about a habit. Answering about a habit requires the work of gathering several events. The gathered result is what comes out as the answer.

Two things happen in the gathering. First, the frequency gets rounded. Second, the exceptions drop out. The three weeks with a gap in them are not inside about once a week.

This structure comes from a property on the side of memory. The memory of individual events and the memory of generalised knowledge are handled as different things. That distinction itself is taken up in how to write from primary information.

A question about a habit summons the latter. Generalised knowledge is the image a person holds of themselves. That image is tidier than the behaviour.

Ask how much of your mail do you read and pretty much all of it comes back. Ask which was the last one you read and a specific message comes out. Where none comes out, that too is information.

If you only want a tendency, a summary suffices. Where it stops sufficing is as material for design. I read all of it yields nothing to design from; I look at the subject line and open up to the third of the day does.

There are cases where a summary is accurate. For simple behaviour with few occurrences, rounding does not happen much. Rounding grows large for behaviour that is frequent and full of exceptions. Even within one interview, then, how far an answer can be trusted differs by behaviour.

The two can be told apart by the grain of the answer. Words like usually, mostly, basically mark a summary. A date or a specific scene marks an event.

There are circumstances on the question’s side too. What do you do is easy to answer. Being easy, the other party can answer without pausing. The room does not stall, so it is easier for the interviewer to keep moving. The easier the question moves, the more processed the answer that comes back. An interview that does not stall is not necessarily an interview that went well.

Change to a one-instance form and the interview takes longer, because recalling takes work. In exchange, the number of instances you can gather falls. Two people asked about events can yield more material than five asked for summaries. Count sits on a different axis from the quantity of material.

Ask Why and a Reason Gets Constructed

The question why did you choose that usually brings an answer back.

The reason that comes back is not necessarily the reason that operated. People do not have full access to the process of their own judgements.

There is an old paper on this. Nisbett and Wilson organised, on the basis of several experiments, the finding that when people report on their own higher cognitive processes they are, in some cases, not observing the actual process and reporting it but constructing a plausible reason (Nisbett & Wilson, 1977, Psychological Review, 84(3), 231–259).

This is a paper on self-report about higher cognitive processes; it is not the claim that all self-report is untrustworthy. Reports of what was done and what happened, and reports of why it was done, receive different treatment.

Put into practice, it comes to something simple. The answer to why is the explanation the person gave to their own behaviour. An explanation is tidier than the behaviour itself.

Ask why did you choose this and because I thought the content suited me comes back. Ask what else did you look at before choosing and the objects compared and the order of looking come out. From the latter, the route of the judgement can be reconstructed.

The worry that you cannot learn the motive without asking for the reason holds. The motive gets estimated from the route. What was looked at, what was dropped, what was chosen. Inside that sequence the material of the judgement appears. The person’s own explanation can be handled as one candidate for that estimate.

Nor does this become an argument that reasons must not be asked for. Record the answer as their self-interpretation rather than as a fact, and both remain usable so long as they are not mixed.

The way of keeping them apart is simple. Record it as they said that. Record it as they bought it because of that and an interpretation gets preserved as a fact. A preserved interpretation wears the face of a fact when next you read it.

The answer about a reason has a use as well. How a person explains their own judgement shows the framework they hold. A framework becomes vocabulary when writing. Used as material for vocabulary rather than as fact, what was heard keeps its meaning.

Proceed without asking for reasons at all and the interview feels clerical, since for the other party it stops being an occasion to talk about their thinking. Here, placing reasons in the second half and giving the first half to events works. Changing the order alone secures both.

The Questions in a Customer Interview Should Specify the Last Single Instance

To avoid summaries, make the unit of the question an event.

The form when was the last time you did that is the shortest question. It specifies a single instance. Specified, the work of gathering does not occur.

What can be asked next is settled too. What were you doing at the time. Where did you see it. What had you been doing before. What did you do after. All of them are events inside a single instance.

What can be taken from one instance is more than what can be taken from a summary: the scene, the order, the trigger, the point of interruption. All of these have dropped out of a gathered answer.

Ask when did you register and last month comes back. Continue with what did you see when you registered and what were you looking for before registering and the route comes out. The route contains the part the person had not put into words.

I do not remember counts as material too. It shows that the behaviour was not an event that stayed vivid for that person.

The point that one instance skews things holds. Gather several skewed instances and a tendency comes into view. Gather several summaries and no tendency comes into view, because a summary has already been averaged. Averaging averages does not restore the original distribution.

There are conditions for behaviour that can be asked about as an instance: it occurs with some frequency and takes a form that stays in memory. Fine actions repeated daily cannot be recalled as an instance. Here the form of tracing the most recent day can be used. Ask what they did from yesterday morning onward in order, and within what can be recalled, a sequence of behaviour comes out.

Asking about one instance leaves room for digging. An event has a before and after, so branches extend from it. A summary has no before or after, so there is nowhere to continue. After about once a week almost no further question remains.

The way of building a question sheet changes accordingly. Rather than lining up items, prepare the first question and the directions in which to extend from it. A sheet of lined-up items is suited to gathering summaries. All the more so once filling in the sheet becomes the goal.

Asking one instance at a time increases the burden on the other party, since it requires recalling. As the burden rises, the talk gets shorter. What was shortened gets covered by taking longer or by cutting items to ask about. Where neither is possible, what gets cut is the items. Keep the items and give up the instance and what remains in the record is summaries only.

In a Room With an Interviewer, Answers Get Matched to the Interviewer

An interview happens in a room with two people. The room itself bears on the answers.

The answering side knows who is listening. Where the person who built the product is asking, the answer is addressed to the person who built it. Negative content gets softened.

This direction of skew has been handled on the survey-methods side as well. Tourangeau and Yan organised the tendency for answers about matters that may be received as undesirable to be under-reported, and the way the size of that under-reporting varies with the format of the survey (Tourangeau & Yan, 2007, Psychological Bulletin, 133(5), 859–883). In formats with a person present, undesirable content gets reported less than in self-administered formats.

This handles bias in survey responses; it did not measure small-scale interviews run by a business owner. What was shown is the direction: the presence of an interviewer exerts a directional influence on answers.

The softening is not necessarily deliberate. It arises naturally in the course of speaking while watching the other party’s reaction. Starting to say something and stopping, lowering the degree, putting the good points first.

The result of an interview therefore contains the influence of the interviewer’s side. The influence cannot be erased. What can be done is to hold on to its direction.

The direction is usually one. Negative content falls and positive content rises. What negative content did come out, then, is to be read as having been stated smaller than it is.

Suppose the sentence there was a slightly hard-to-follow part comes out. This sentence is the form after softening. Continue with which part and the concrete comes out. Do not continue and it stays in the record as a small remark. The record that remains gets read later as a lighter problem than it was.

Having a third party ask is one available move. Part of the softening falls. In exchange, you cannot ask deeply. A third party does not know the substance of the product, so they cannot see where to dig. Which to take is settled by what you want to know.

Softening arises less in some settings: a long-standing relationship in which the other party speaks frankly. Even here the direction remains. The more goodwill there is, the harder negative content becomes to say.

One condition attaches to the manner of asking. Show in words, at the outset, that negative content is welcome. Place the parts that did not suit you are the more useful ones first and the width of the softening falls a little.

Show welcome and negative content can now come out in excess, since the other party tries to meet the expectation. So the showing happens once, and after that the reactions are kept flat. The strength of a nod also reaches the other party as part of the material.

The Moment You Settle Whom to Ask, What Can Be Heard Is Settled

Selecting whom to ask is not the stage before the interview but part of the interview.

The range of experience held by the person selected becomes the range of what can be heard. From somebody still using it, you can hear the conditions under which they continued. From somebody who stopped partway, the conditions under which they stopped.

Without asking both, only one side’s conditions accumulate. And the easier side to gather is the former, since people still using it are more likely to agree.

This skew takes the same shape as the sampling problem handled in customer testimonials. Where the visible set has been extracted under a particular condition, reading that set correctly still does not tell you about the whole.

Settling the audience itself is handled in how to build a persona: the audience is an output of the design rather than an input, and can be narrowed only after the criterion of value has been settled. In interviewing, this order shows up in the judgement of whom to ask.

Ask while it is unsettled what you want to know and you end up asking whoever you can. People you can ask are people close to you. Answers from people close to you carry a wide softening and similar conditions as well.

There is also the order of asking first and settling the audience from there. The answers of the first few people, however, settle the design of the later questions. Once the design is settled, what can be heard beyond it is settled too. The skew of the first few extends to the whole. The skew becomes invisible at the moment it moves into the design, because no trace of it remains inside the question sheet.

At some stages the audience cannot be settled strictly: just after starting, when even who is using it is unclear. Here you interview while recording that it cannot be settled. With a record, the skew can be corrected afterwards. What cannot be corrected is the case where the inability to settle went unwritten.

There is no rule of thumb for how many people to ask. The point at which the same answers begin recurring is one marker. Where the marker arrives early, it indicates that the audience is too narrow. The same answers line up not because the whole has lined up but because only people under similar conditions were gathered.

Settle the audience first and you cannot hear from people outside the settled range. The next audience is sometimes inside that outside. So the form of asking one person outside the range, once, works. One is enough. One instance from somebody outside teaches you where the edge of the range is.

Place the Record Before the Interpretation

What was heard gets kept in the form in which it was heard.

This is the reason for transcribing. Organise from memory and only what stayed vivid survives, and at that point an interpretation has entered. An interpretation that has entered cannot be removed afterwards, because no record survives of which part was interpretation.

Keeping what was observed apart from how it was assessed is the discipline set out in how to write from primary information. In interviewing, that discipline applies at the stage of the record.

Concretely, two things get written in separate places. What the other party said. What you thought from it. Mix them in one place and, on rereading later, the two cannot be told apart.

Where they cannot be told apart, you remember your own interpretation as the other party’s statement. And you begin picking up, in the next interview, only the statements that support that interpretation. The first interpretation settles the design of the second interview.

Suppose the statement I just forgot to use it appears. What gets recorded is that sentence. Write needs a mechanism for habit formation and that is an interpretation. The interpretation can go in the adjacent column. So long as it sits adjacent, it can be detached later.

The point that material is unusable without interpretation holds. Interpretation is necessary. What matters is the order. Take the whole record first and interpret afterwards. Interpret while listening and you dig into statements that fit the interpretation and let the ones that do not slide past.

Digging in on the spot is separate from interpreting. Asking what kind of situation was that about a phrase that caught your attention is the work of thickening the record. Asking so, you mean this as a summary is the work of getting the other party to endorse an interpretation.

There are occasions where confirmation is needed, to prevent mishearing. In that case, repeat the other party’s words as they stand. Paraphrase and your own vocabulary enters. Vocabulary that has entered stays in the record as words the other party endorsed.

Mixed-in vocabulary becomes hard to spot later. Where the record contains words you are used to using, that is the form after paraphrase. Where a word the other party would not use appears in the record, that passage becomes something to check again.

Take a thick record and the work of organising increases, since there is more to reread from an hour of interview. The way to reduce it is to narrow what you want to know to one thing before asking. Narrow it and the record stays thick while the range to read gets settled. The work is settling how to read rather than trimming the thickness.

The Heavier Information Sits With the People You Cannot Interview

Interviews are possible only with people who agreed.

Three kinds of people do not agree. People who did not buy. People who bought and did not use. People who used and left. All three are hard to reach.

And the information these three hold is of a kind the people who agreed do not hold. Why it was not chosen. Where it stopped. What did not suit.

Repeat interviews, therefore, and the material leans to one side. That it leans does not appear inside the material. Not appearing, there is no occasion to notice the lean.

The remedy is not complete. Two things can be done. First, state the range you have not covered. Second, build a route by which at least a few can be reached.

There is the form of asking somebody who registered and never opened anything a single question. What were you expecting when you registered. With one question, an answer is sometimes given.

The sense that asking somebody who left is awkward remains. Conditions attach to the manner of asking. State that it is for improvement, narrow it to one question, and add that they need not answer. Mix in wording that urges a return and it becomes an announcement rather than an interview.

The existence of people you cannot reach need not be treated as a defect, since no sample is ever complete. What becomes a problem is only using it while believing it is complete.

And the range you cannot reach can be partly filled by other material. Where it stopped is sometimes visible from the records. The reason is not, but the position is.

The position alone lets the next question be raised. Where many people stop at the same place, something is there. What is there can often be learned by asking somebody who got that far. You cannot ask the people who left, but you can ask people who passed the same position. The position does not substitute for the reason, but it does give the question somewhere to sit.

Stating the range takes one line. Write asked five people who are still using it. Written, the you who reads that record six months later will not mistake the range. Unwritten, the record survives as the result of asking users.

Hold the unreached range in view and the conclusions drawable from the interview get weaker, since you can say no more than the people using it say this. What is weak is the accuracy, and that is not a defect. To make a strong conclusion, you widen the sample. Say it strongly without widening and the conclusion extends beyond the range.

Reading, at the Organising Stage, Whether the Record Is Serving as Material

Whether an interview is serving as material for the design can be checked in three ways. All can be carried out at the organising stage.

The first confirmation is to see how many descriptions containing a date and a scene can be picked out of the record.

If none can, what you heard was summaries only. A record composed of usually and mostly is that person’s self-image, not a record of behaviour. In the next interview, the unit of the question moves to events.

The second confirmation is to see whether what the other party said and what you thought are written separately.

If they are not, which is which becomes indistinguishable with time. A record you cannot tell apart can only be used as material supporting your own hypothesis.

The third confirmation is to write out the range you did not reach.

People who did not buy, did not use, left. Count how many of each you reached. If it is zero, all the interview lets you say is something about the people who continued.

How many people and how long sit outside all three. Quantity is a condition, and lies on a different axis from whether it is serving as material.

One question survives all three: whether what you heard matters to the design. A record satisfying all three can still fail to bear on the design. That judgement can be made only at the stage before asking. What, if learned, would change the next judgement. Unless that is settled, what you heard ends as it was informative.

How many people, how many minutes, how many questions to prepare. No numbers are attached. People and minutes are precisely the vocabulary of the summary side. Place a number first and count takes priority over the time spent digging one instance. The allocation settles from what you want to know and the other party’s circumstances.

That about once a week, I suppose is a product of processing stays invisible in the room, because the other party answered sincerely and the interviewer heard what they wanted to hear. The processing shows plainly when the same person is asked again about a single instance. The difference that emerges there comes from the unit the question specified, not from the skill of the asking.

A record depicting the image the person holds of themselves is material on their framework. A record depicting the events that occurred is material for design. Which of the two you are holding comes out of the three confirmations at the organising stage. What you then use it for is settled by the design that follows.

The stage of publishing what you heard is customer testimonials, and settling the audience is how to build a persona. The stage of settling what to write from the material you took sits next door, in what to do when you do not know what to write. Taking the event that happened rather than the judgement the other party stated is because the material is only there. The side of what gets extracted from someone else’s words is treated in the design of verbalisation.