A converse situation is very often encountered, in which we know about the target sign system, and seek to infer properties of signs in the source system from their images in the target system. This occurs, for example, when we try to understand a poem, an equation, a sentence, or indeed, anything at all. Let us call this the inverse problem, as opposed to the direct problem of design.
We address questions about the nature of translation between sign systems, and the reasons for preferring one translation to another, by studying maps from signs in one system to "represent" signs in another system. These maps are called semiotic morphisms, and are made precise in Definition 2 of the paper [B]. Examples include metaphors, analogies, etc., as well as representations in the more familiar user interface design sense. Just as we defined sign systems as theories rather than models, so their mappings should translate from the language of one sign system to the language of another, instead of just translating the concrete signs in a model. This may seem indirect, but it has important advantages over a model based approach to representations, as we discussed in Section 2.
A good semiotic morphism should preserve as much of the structure of its source sign system as feasible. Certainly it should map sorts to sorts, subsorts to subsorts, data sorts to data sorts, constants to constants, constructors to constructors, etc. But it turns out that in many real world examples, not everything can be preserved. So we must allow all these maps to be partial. Axioms should also be preserved - but again in practice, sometimes not all axioms are preserved. The extent to which things are preserved provides a way of comparing the quality of semiotic morphisms (this is the point of Definition 3 in [B]).
As a first example of a semiotic morphism, suppose we want to represent time-of-day (TOD) in the little screen W. Clearly there are many ways to do this; all of them must map the sort time to the sort screen, map the constructor 0 to some string of (less than 25) strings of (less than 81) characters, and map the constructor s to a function sending each such string of strings to some other string of strings. There isn't anything else to preserve in this very simple example except the axiom, which however is very important.
Recall that the items of abstract syntax in TOD are strings of up to 1439 s's followed by a single 0. One simple representation just maps these strings directly to strings of strings of s's such that the total number of s's is the same. Let N(t) be the number of s's in some t from TOD. Let Q(t) and R(t) be the quotient and remainder after dividing N(t) by 80. Then there will be Q(t) lines of 80 s's followed by one line of R(t) s's. This is guaranteed to fit on our screen because Q(1439) = 17 < 25, and R(t) < 80 by definition. Although this is a digital representation, it is so detailed for humans that it is essentially analog, in the sense that after becoming familiar with it, a user gets a "feel" for the approximate number of (these strange 80 minute) hours, and the number of minutes that remain. (The maximum number of items that most humans can "subitize", i.e. immediately recognize without counting, is 4 or 5.)
One obvious representation just displays N(t) in decimal notation, giving a string of 1 to 4 decimal digits. This is very different from our usual representations; but we could imagine some strange culture that divides its days into 18 "hours" each having 100 "minutes", except the last, which has 40. (Actually, this is less strange than what our culture actually does with months!) Here N(0) is 0, and s just adds 1, except that s(1439) = 0.
A more familiar representation is constructed as follows: Let N1 and N2 be the quotient and remainder of N divided by 60, both in base 10, with 0's added in front if necessary so that each has exactly 2 digits. Now form the string of characters "N1 : N2". This is the so-called "military" representation of time; let's denote it by M. Then M(0) = 00:00, and of course you know how s goes. The spoken variant of military time has the form "N1 hundred N2 hours" (unless N2 = 00, in which case it is omitted). The use of "hundred" and "hours" may seem odd here, because it isn't hundreds and it isn't hours! - but at least it's clear - and that's the point - the word "hundred" functions like the colon in the written form, and it has a distinctive sound. Notice that this representation has been defined as a composition of N with a re-representation of TOD to itself.
I think you can now see for yourself how to construct other representations of time as semiotic morphisms, including even the "analog" representation of a clock face. Here 0 has both hands up, satisfaction of the axiom follows because something even stronger is true, namely s(719) = 0, which is built into the circular nature of this geometrical representation. You might think that satisfying this stronger axiom means that the representation is poor - but is it really? Think about why representation modulo 720 works, but (say) modulo 120 or modulo 300 do not work; what about modulo 360? Why? Hint: Think about the context of use of the representation.
Now let's imagine someone who has to design a text editor based on the screen W; part of their job is to construct a semiotic morphism E from TXT to W. One issue to be addressed is the correct placement of spaces, so that words are always separated, and sentences end with a period followed by two spaces; another issue is the initial capital letter. The designer will also have to decide what to do about words that want to go past the end of a line, e.g., to wrap around, hyphenate, or fill in with extra spaces. The limit of 24 lines will also pose problems.
If instead of TXT we have PS available for input to W, then a designer can consider some more sophisticated issues, such as automatic insertion of commas. It may also be interesting to think about representations from PS to TXT. The most obvious and familiar one just gives the unparsed form of the parsed sentence; but again we can imagine adding some punctuation.
An interesting semiotic morphism from the experimental work of [9] is the correspondance between the physical order of lights on the experimental device, and the order of clauses describing those lights (it turned out to be narrative order).
Finally, a somewhat spectucular example of a semiotic morphism is the LaTeX2HTML program used to produce the original version of the document that you are now reading. LaTeX2HTML takes a LaTeX source file and produces an HTML website that preserves the structure of the source file. However, lIt is far from perfect, and a fair amount of editing was required to get a good quality first version of the current website; for example, subsections were always given separate pages, even when they were only a few lines long. A similar program for Windows, called TTH, has somewhat better quality.
To begin to formalize and extend the intuition that Peirce's 3-fold classification of signs reflects the "naturalness" or "immediacy" of their signification, notice that semiotic morphisms may preserve aspects of a given sign system to a greater or lesser degree, and that this degree affects the quality of the representations which it provides. The structure of the entire system of related signs must be considered in determining what makes one representation better than another, because the structure, priority and level of signs must be considered in constructing their representations.
The fuller exposition of this material given in [B] (further developing ideas from [9]) includes detailed definitions of what it means for a semiotic morphism M to be level preserving, constructor preserving, priority preserving, attribute preserving, and structure preserving. It is important to notice that semiotic morphisms need not be totally defined; that is, each of the functions denoted M can be undefined on some of what is in S1. For example, there need not be any representation in S2 for some signs in S1; in particular, some components of M could even be totally undefined, i.e., it could be an empty function.
It might at first seem obvious that a good structure preserving semiotic morphism M : S1 -> S2 should faithfully represent all of the semiotic structure of S1 in terms of that available in S2. But if the resulting structures in S1 are too complex, then they may be hard for human beings to understand. For example, if S1 consists of parse trees for English sentences and S2 consists of the usual "printed page" text format, then it is possible to translate all the syntactic information that is available in S1 into structures in S2 with so-called "phrase structure" notation, which uses brackets to delimit phrases and uses subscripts on brackets to indicate the class of phrase is involved. For example, the representation
may be useful for some purposes, but it is clearly not optimal for others. There is a trade-off between the degree of structure preservation and the complexity of the resulting representation.
It may be that neither of the morphisms M, M' preserves strictly more structure than the other, or that one preserves more structure but produces more complex representations. For example, M might preserve more levels while M' preserves more attributes. The experiments in [9] show that preserving levels is more important than preserving priorities, which is more important than preserving attributes. They also show a strong tendency to preserve higher levels at the expense of lower levels. This was unexpected, because of work by Rosch and her students explicating the notion of "basic level" for lexical concepts (e.g., "bird") [14, 15]. For natural language, the sentential level was long supposed basic, but much recent research emphasizes the discourse level. In short, structure is more important than content; it is also easier to describe precisely.
Note that when we consider preserving various structural aspects of signs, we are dealing with various forms of Pierce's notion of iconicity, i.e., with properties of the sign itself. It is more complex to characterize Peirce's notion of indexicality in our framework, because it involves relations that hold between a sign and its representation, which takes us into a larger semiotic space that includes both; this can be done, but we will not pursue it further here.
When we see a complex sign, such as a diagram in a technical report, or an abstract for a lecture, we assume that it is intended to convey some information, and we then try to infer what it is that the author intended. In the language of this paper, we seek to infer a source sign from the given target sign. The difficulty of course is that we know little or nothing about the source sign system, or about the semiotic morphism (indeed, it is doubtful that such a source sign system really exists in the usual sense). Hence we must try to infer a systematic mapping from an unknown space of intentions to another space of displays, such that the observed display is the result of applying that mapping to the inferred source sign, under the assumption that the author carefully chose the given sign as an appropriate way to represent certain information. An important aid to this inference process is the assumption that the author has used a systematic mapping which will apply to other signs from the same space, not just the one that we happen to have seen; this assumption allows us to eliminate many irrelevant possibilities when inferring the author's intent. That is, as readers, or more generally, as interpreters, we assume existence of a semiotic morphism (or more precisely, of the intent that a semiotic morphism formalizes).
Inferring a sign in the source system from which a given sign in the target system arises is an instance of what we called an inverse problem; it is the essence of semiosis. We called the converse, to design a sign in the target system, a direct problem. It might seem that inverse problems are more difficult than direct problems, because we have to infer both the semiotic morphisms and the source space, whereas a direct problem often only requires constructing a morphism (and perhaps fine tuning the target space). However, the opposite seems to be true in practice! Ordinary humans infer the meaning of typical signs (e.g., complex sentences) almost effortlessly and instantly, whereas design (e.g., user interface design, technical writing, automobile body design, etc.) is a demanding task for professionals. In my opinion, it is not so hard to do bad design, but good design is inherently demanding, difficult, and creative, because the result must be easily used by the target audience, i.e., the designer must design the morphism so that the inverse problem is as easy as possible for the intended users. (Actually, things may be a little more complex, because users may use the assumption that an interface was designed to be easily understood as part of their process of understanding, so designers may need to consider what users will think that they have been thinking!)
Notice that we are not addressing art in this note, but rather design, in the sense of craft. Art is even more inherently demanding, difficult, and creative than design; for example, it often involves deliberate transgression of preservation properties (among other things). Whereas design is typically utilitarian, i.e., done for a practical purpose, contemporary art is typically not utilitarian, even though traditional art tends to be utilitarian (more precisely, this distinction may not even exist for traditional art forms.) Also the distinction between art and craft is not so rigid as this discussion suggests.
I have noticed in my own design practice that I often have creative leaps, i.e., design ideas just occur, which I then try to understand in more formal ways, e.g. via semiotic morphisms, although if I am stuck, it may help to think more formally about the structure of the source space, preservation properties, etc. The creative process is to some extent unpredictable and uncontrollable; this is more true of artistic creation than of design, but it holds for both. The best designs often seem both surprising and obvious, and they also often seem to come suddenly out of nowhere, but usually after a lot of hard work.