"How can one think what is not the case?" is a typically troubling question in philosophy. We know perfectly well that it is possible to think what is not the case, but we have trouble explaining exactly how is it possible, as if there were some mental mechanical explanation that we haven't quite understood.
Since we can think of things and facts, we mistakenly assume there must be "objects of thought," and wonder how something that does not exist can be such an object. Perhaps the objects of thought are "shadows" of things or facts: mental objects that somehow correspond to things or facts. But how can I recognize a certain shadow to be the shadow of a certain fact? It seems there is some act of interpretation by which I interpret the shadow as being a shadow of a certain fact.
We could imagine a convention according to which we follow the stem of an arrow rather than the point: there must be an act of interpretation involved in reading "—>" as "go right" instead of "go left." In turn, this act of interpretation could be represented as a sign, perhaps as another arrow showing that the previous arrow means "go right." But then this second act of interpretation stands in need of interpretation, raising the question of where the chain of interpretations stops.
Our inclination is to say that what a sign says can be interpreted, but that what the sign means does not require interpretation. Though we may be deceived by grammar into thinking that "meaning something" and "saying something" are analogous, what something means cannot always be represented by signs. When I say, "I'm delighted to see you," whether I mean it or not is determined by my tone and attitude, and not by certain words in my head.
The notion of a "shadow" of a fact comes from the assumption that a fact must be present in our minds if we are to say we are thinking of it. But this assumption leads to the unsolvable difficulty of how the mind is able to interpret this "shadow" as representing a particular fact. The assumption that shadows of facts exist in our minds comes from a particular form of expression. We say things like, "when I said 'Napoleon,' I meant the man who won the battle of Austerlitz." By this we mean that we said a word, and partially defined that word by something unspoken, something "in our heads." There is nothing inherently bad about saying we something "in our heads" as long as we recognize that this expression is metaphorical.
Wittgenstein does not say there are no or processes associated with thought or meaning, he just dismisses the assumption that there must be complicated mental states. No distinctive activity of meaning what we say necessarily underlies all speech. Wittgenstein calls "meaning" an "odd-job" word, one that serves a number of different important purposes. We will have no luck if we look for the one distinctive process of meaning that exists in the peculiar medium of the mind.
In Wittgenstein's discussion of an "object of thought" there is a subtle criticism of the views he expressed in his earlier work, the Tractatus Logico-Philosophicus. In that work, he describes a thought as a "logical picture of facts." He suggests that a thought can correspond to reality because both share a common logical form. Both reality and thoughts are made up of elementary units combined in different, complex ways. Thought and reality have the form of these complex combinations in common. We can think of objects that do not exist because we can combine elementary objects that do exist in ways that do not correspond with reality. This view, expressed in the Tractatus, provides one answer to the troubling question, "how can one think what is not the case?"
Here Wittgenstein criticizes his earlier view and any other theory of thought that tries to answer this question, saying that such ideas simply build complex theories around a mistaken assumption. This assumption, roughly, is that to think of something, or to mean something, or to fear something, we must have that something present before our minds, existing in a mental space. Wittgenstein says we are drawn into assumptions such as these because we are misled by particular forms of expression. Words like "mean" do "odd jobs." There isn't one single meaning of the word "mean," but rather a family of different uses. We fall into philosophical difficulties when we try to say what "meaning" is, and then rely too heavily on one particular use of "mean" in formulating our definition. This particular use, or "form of expression," leads us astray.
Wittgenstein gives us an example of the person who says "Napoleon" and means "the man who won the battle of Austerlitz." We think that whenever we speak, our utterances are accompanied by a parallel meaning (such as "the man who won the battle of Austerlitz") that can be expressed in signs. We then think that what we mean when we speak is like an inner voice, and come to think of meaning as something that exists in the mind. Some readers might object that they don't say one thing and think of another meaning expressed by an inner voice. Wittgenstein would have two replies to that objection. He states that in ordinary use of language, we are not led astray by expressions such as "having a thought before my mind." Such expressions are perfectly all right so long as we do not try to build a philosophical theory upon them. Second, he would agree that the idea of an inner voice is a simplified way to describe what happens, and that no serious philosopher would reason quite like that.
This admission that serious philosophers would take issue with his ideas characterizes Wittgenstein's later philosophy. He very rarely engages with the ideas of a particular philosopher. Instead, he deals with certain basic assumptions upon which more complex philosophy is built. Wittgenstein emphasizes that complex philosophical theories do not amount to much if they are simply refinements on an initially faulty assumption. Thus, instead of trying to refine already refined theories, he tries to draw us back to the initial assumption that leads us astray. No one claims that thoughts are literally "shadows" of facts, but this is an expression of an idea that has taken form, in one shape or other, in numerous philosophical theories.
The question of how one interprets an arrow sign plays a peripheral role in this discussion, highlighting the problem with claiming that there is a correlation between what one says and what one means. This question of how we interpret will become increasingly significant to Wittgenstein in his later philosophy, as he begins to ask how we know how to follow a rule.