If you ask me whether I'd rather go to a party tonight or stay home to relax, I'd probably tell you I'd rather stay home. But overwhelmingly, whenever I do choose to go out, I have a nice time seeing friends and I'm glad I left the house. If you ask me about my ideal work environment, it's a quiet room where I can be left alone. But if I look back over my biggest accomplishments, it's often been in busy environments surrounded by bustling people.
What I think I prefer and what I actually prefer are often two different things. And this is not unique to me.
Charles Duhigg has a fun example of this in The Power of Habits: men tend to state a dislike for Celine Dion’s music, but don’t change the station when her music comes on the radio. Which is why I'm very interested in the pervasive use of surveys, especially when they ask people about what they are about to do. We poll Americans about which party they are going to vote for, and then are surprised when the outcome of the election is drastically different.
To be sure, surveys do have a place. But they are effective for the same types of questions I could answer with confidence. Ask me about what I have already done, and the answers start getting much more confident. For example, ask me about which political party I’ve donated money to, and that answer is much more useful. There is an effect with regards to recency as well. Ask me what I ate for breakfast last Thursday, and I’ll struggle to remember. This is why Mihali Cziksentmihali’s Experience Sampking Method, of having people immediately record their feelings at random intervals, was so powerful in coming up with this theories of flow.
It’s difficult, though, to always gather insights from past behavior. Which is why I think speculative surveys are still common, as they are “better than nothing.” But I wonder if this is true. Perhaps it’s better to acknowledge that we really don’t know about something. And, on the other side, be very sceptical of any conclusion that depends on speculative survey data for its results.