Getting beyond the link primitive:

Expressing textual structures
in terms of link structures

Paul Rohr
Nov. 13, 1991

Chunk Models [me... M85]

"Chunky" models of hypertext can be ranked by their structural complexity.

The simplest "chunky" model of hypertext allows only node-to-node links. All you get are chunks of text and links between them.

Thus the tendency is to fragment your information as much as possible, using links to represent all structural relationships.

Example: HyperCard

A slightly more complex model allows for point-to-node links. Here, specific spans of text within nodes are allowed as source-points for links.

This allows for somewhat larger nodes, and for a partial ordering of information. (Each point expands to a larger chunk.)

Example: Storyspace

The next most complex model allows for point-to-point links. Here, the destination as well as the source can be a point.

By implication, this model can derive a reduction to the first one, since there is an implicit (albeit far less detailed) relationship between chunks which are the site of linked points.

Example: InterMedia

Linearity [me... L2]

Hypertext is often touted for its ability to work against the "linearity" of text. In other words, there is a preconceptual notion that text is a linear sequence (or string) of words.

"Chunky" theorists represent the conceptual complexity of the discrete ideas woven into that "sequence" by chopping it up into smaller, more manageable chunks of text, and then explicitly re-linking them. The chunks all remain (locally) linear, but they're each folded up into a separate node and linked together.

Thus, for "chunky" theorists, the chief problem is to decide how to divide up long strings of text into shorter ones whose linearity is not a problem. This is known as the "fragmentation problem".

For large amounts of text, this choice of element granularity can become very important, since it becomes increasingly difficult to find one's way around in larger and larger spaces of such locally-linked chunks. This is known as the "disorientation problem", and is endemic to collections of data whose mass of detail greatly outstrips the amount of general information available for structuring, arranging, and locating specific instances -- whether in relation to one another or to those larger referential structures.

Unfortunately, at this (low) level of theoretical complexity there's simply not that much structure around to use as a lever. Basically, you have three structural elements to work with: nodes, links, and node-link interactions. The first two can be named and/or assigned to known classes of types, information which can then be used (via querying) to group similar nodes, and filter out unwanted ones. On the other hand, node-link interactions can be used to either generate maps and overviews (both locally and more generally) or to calculate proximity measures of the clustering of and distance between nodes. Finally, any text affiliated with any such structures can also be the subject of free-text searches for occurrences of various words or phrases (another kind of filtering).

However, even in combination, these structures just don't provide all that much leverage for navigating large populations of chunked information.

Creamy Models [DeRose, Durand 87]

In contrast, "creamy" theorists insist that the problem is that there's simply too much "chunking" going on. Instead, they tend to focus on how to retain the structural power of large expanses of text -- particularly as presented in the highly-evolved formats of printed books and scholarly articles.

In other words, "creamy" models assert that the "largest natural unit" of text is the document, and it must be represented at that level directly, as well as being broken down into smaller substructures.

OHCO [DeRose, Durand, Mylonas, Renear 90]

Although these substructures could be though of as also occurring in sequence -- thus forming a "chunkier" sort of "linearity" -- a typical "creamy" theorist instead emphasizes the presence of other sorts of structures, and especially hierarchies.

In the terminology of the two papers cited, a document is an Ordered Hierarchy of Content Objects (OHCO), a term which is intended to suggest how the linear sequence of textual exposition (at the bottom) is an ordered traversal of the hierarchy (above it).

Note the reciprocal relationship between segmentation and aggregation here.

Implicit in the notion of a hierarchy is the ability to "roll up" smaller substructures to higher levels of aggregation, thus affecting the level of granularity of segmentation.

Likewise, an equivalent transformation yields:

Note, however, that these transformations can be derived as an algorithmic result of the overall structure, and thus need not be explicitly stored.

As always, the storage-vs-recalculate tradeoff will depend on the particular speed and cost of available resources.

Beyond Creamy [me... M83]

The "creamy" doctrine that texts should be represented as full documents can be extended in a number of ways. After all, the composition or aggregation links used to indicate internal hierarchical structure within texts are not the only sorts of links possible.

Indeed, the most hypertextual of these are the external links between documents, in the forms of citation and allusion (among others).

Once again, there is an implicit ability to aggregate here, reducing the detail of numerous point-to-point links to a composite text-to-text link.

Knots [Harpold 91]

However, contemporary theorists of textuality understand these intertextual (or transtextual, depending on the theorist) relationships in a somewhat different -- and perhaps more radically hypertextual -- way. Instead of thinking of individual texts as distantly separate chunks which are connected by thin links, they might choose to think of each text as a winding thread of discourse which is only locally linear (in its exposition) and which crosses other textual threads from time to time.

Harpold refers to the (potentially tangled) site of this crossing as a "knot".

Threads [me... L2]

In the very simplest class of texts, there may be only one such crossing of any two textual threads, although any single text probably crosses a number of others. For example, conventional scholarly articles often include an initial section which reviews and summarizes prior results on a given problem, with a brief gloss for each citation.

When the author is only interested in the main results of an article, she may choose to cite the entire thing, or she may refer to a specific aspect of that work, localizing the citation to a particular portion of the article.

However, the supposedly rigid "linearity" of these threads comes into question as soon as multiple citations of the same work are allowed.

In such a case, there is no way to think of both threads (that of the text and that of the subtext) as being simultaneously linear. Depending on the reader's perspective, one of the threads may be thought of as linear, but this necessarily implies that all other threads entangled with it are not.

Note that in this example, the order of the cited spans is the same in both texts. While this assumption simplifies the picture, in the more general case we often find the order of exposition varying greatly between text and subtext -- even in the most faithful of close readings! (Indeed, this variation is one of the chief advantages of such readings.)

Imagine, then, the complex "geometry" of any text in which a number of such threads are interwoven, and the conclusion becomes obvious: only within the immediate locale of a particular interaction can any two threads be considered simultaneously linear.

Zones [me... M33]

Of course, there are many other sorts of links besides citations and allusions. For example, anything which might be indexed can be treated as a link, since there is both a source (the index entry) and a destination (the occurrence of the indexed feature within the text). Each individual link points to a specific span of text, and it's important to specify such links and spans at the smallest applicable level of granularity. Still, the usefulness of any individual link, considered in isolation, is confined to the simple operation of following it. In contrast, if we consider the more complex structures which can be built up out of links, then far more powerful operations become possible.

As the diagram indicates, link structures can divided into two general classes, "threads" of occurrences and schemes of "zones", depending on how they cover the text. Generalizing the notion introduced in previous pages, we define a thread as a sparse set of discrete occurrences -- specified in terms of textual spans -- which have something in common. From another perspective, a thread can be seen as a set of links from a shared feature (in the analogy above, the indexed term) to its various occurrences. Note that aggregate collections of threads (for example, all terms in a domain) also form threads, albeit less sparse ones.

Alternatively, we can obtain a different sort of link structure by dividing a text up into a series of contiguous zones which completely cover it. Here, each zone is a span of text, labeled according to some classification scheme.

Zone Schemes [me... K41]

Threads and zone-schemes can both be thought of as ordered aggregations of a set of linked spans. However, there the resemblance ends. Whereas each occurrence on a single thread points to the same shared description (at some level), zone-schemes are more complex. Of course, each zone is a span of text with a label linked to it, but it would be senseless for all zones in a scheme to point to the same label. (A zone-scheme covers the entire text, which implies that the entire text would be covered with spans all pointing to the same label. In other words, there would only be one zone, spanning the length of the text, which in no way discriminates particular locations within the text.) Thus, we can further classify zone-schemes by the systematicity of their zone-labelling.

On the one hand, we can have a consistent set of alternating zones which cover the text in an intermittent fashion. Within such a zone-scheme, recurrences of any single zone could also be thought of as an isolated thread. Still, in the aggregate structure formed by merging the entire set of related zones, this thread-like nature disappears into the mosaic of the overall pattern.

On the other hand, if zones don't recur, but follow one after another in continuous sequence, then we get a quite different kind of zone-scheme. Here we find that a number of continuous zone-schemes can be specified at different levels of granularity, and synchronized to form a hierarchy. Unlike the intermittent case, continuous zone-schemes thus decompose directly into either links or other continuous zone-schemes, rather than threads.

That having been said, we should also note that arbitrary sets of (less organized) zones can, in principle, be treated as (dense) threads.

Algebra [me... M86]

As the partial taxonomy on the previous page indicates, there are a wide range of embedded and interpretive textual structures which can be expressed in terms of the "thread" and "zone" link-structures. With those distinctions and examples in mind, we can now proceed to briefly examine the structural interactions supported by such a system.

Insofar as threads are sets of occurrences of textual features, they are amenable to common set operations (union, intersection). However, the applicability of these operations is restricted by the fact that they presuppose an ability to discretely identify equivalent occurrences when deciding whether to include them in the resulting set. Since the occurrences in question are actually spans of arbitrary length, then in most cases the less strict -- and equally less familiar -- segment operations (overlap, containment) will need to be substituted for the usual equality condition. In other words, strict set operations can only be reliably used to add and remove occurrences from related threads, since otherwise, there is no simple guarantee that spans on the two threads will be either exactly equivalent or completely disjoint.

Likewise, these less strict forms of intersection can be applied to zone-schemes, as well as threads. While overlap operations on pairs of threads, zone-schemes, or any combination thereof always result in either an empty set or a thread (as per the note at the bottom of page 8), it is necessary to specify the desired level of granularity of that resulting thread. The analogous requirement for containment operations is even stricter, since that operation -- no matter how it's defined -- is implicitly *not* symmetric.

Finally, the directed operations (expansion, refinement) can be used to translate any thread in terms of any zone-scheme, thus changing the density of the thread.

[me... K42]

Since a full formal presentation of all of this material still remains to be worked out, the following assertions are offered without proof:

According to the presentation above, the following assertions are false, but they suggest that another operation may be needed:

[Raymond, Tompa 88]

When compared with current "chunky" link-based hypertexts, the full-scale texts that scholars work with every day provide a much richer mix of structural cues and hooks for manipulating them. Anyone who hopes to supplement -- or, indeed, eventually supplant -- paper texts with electronic versions and variants thereof should, of course, plan to provide additional functionality in the new medium (such as active links) which current media and methods cannot easily supply. Yet first these same designers must ensure that in the process of enhancing existing texts and textual methods their current level of functionality is not destroyed or degraded. Clearly the radical expedient of chunking and re-linking texts will have trouble satisfying this standard.

[me ... BA paper, section 3, a lot fuzzier]

My argument here and elsewhere has been that (1) the link primitive itself can only go so far to ameliorate this problem, (2) direct segmentation of the text in any form is equally problematic, and (3) the abstract link structures described here can be implemented externally, by means of the span abstraction, in a sufficiently non-destructive and flexible way ...

... provided a suitable index structure [cf. Bentley, McCreight, and Chazelle] can be found to support that abstraction.

In addition, this approach has the decided advantage of maintaining the "smoothness" of any "locale" in the text, while constantly allowing the reader to expand, explore, and refine the inter-related details of any number of threaded contexts....

[me... M83]

As the partial taxonomy on the previous page indicates, there are a wide range of embedded and interpretive textual structures which can be expressed in terms of the "thread" and "zone" link-structures. It is important to remember that these structures are built up out of links, because this allows them to be completely removed from the body of the text, yet re-applied as needed. Once a canonical span abstraction has been defined (such as the one described in my BA paper), the link anchors no longer need to be embedded in the text. Thus, an arbitrary number and variety of link structures can be separately defined in relation to a given text or set of texts, with specific structures then "layered over" various locales of the text(s) as needed.

The rich interaction of such structures is what makes the TextBase environment so powerful.

In the simplest case,

Although there is quite a bit of coordination within an individual instance of either class of zone-schemes, the same is not true for the interaction of different zone-schemes. The requirement of consistency no longer applies (dominates) outside these structures. Overlaps, rather than strict containments, are the rule.

In principle, any two intermittent zone-schemes will clash, as will any two hierarchies of continuous zones. However, this is a good thing.

The actual interaction of a number of different structures, classifications, etc. is what forms the detailed context of any given region in the text.