poems that change

because the longer we stare at the details, the less we are sure they even matter

For a little over a year now, my work as a composer and writer has been heavily involved with indeterminacy and algorithmic processes. These processes range from procedures using coins and spreadsheets to sophisticated¹ programs that generate distribution-ready documents at the push of a button.

My most recent project in this realm is a collection of mutable poems which are automatically generated and shared through a website - we accidentally imagine. I call them mutable because they change, but at the same time they aren’t completely random collections of words either. Each poem exists as a complex series of probability distributions and generative systems with fixed parameters. Each time the webpage is visited, a program is run which then uses these probabilities to generate new versions of each poem. Each poem has its own set of fixed probabilities and behaviors, giving it a unique character, so to speak.

Let’s take a look at the behavior of one of the poems across a few versions too see what this really looks like: ²

Notice the similar word frequencies, the way some words tend to follow others, the way the spacing and dashes between words has certain tendencies. Each of these elements, and many more, are controlled by discreet systems which use fixed probabilities to generate novel details. While the underlying systems and probabilities remain the same, the details of realization always change.

Let’s open up the configuration for this particular poem and see what I mean by fixed probabilities. (By the way, all of this code is open source and you can check it out here)

{
'filename': 'agains_the_wayingsong_a_we.txt',
'name': 'fourteen',
'immutable_id': 14,
'distance_weights': {-3: 14.1,
                     -2: 17.2,
                     0: 19.8,
                     1: 168.7,
                     2: 19.5,
                     3: 17.2,
                     4: 22.5,
                     6: 19.4,
                     10: 13.5,
                     16: 6.5},
'mutable_chance': 0.59,
'position_weight': 16.82,
'x_gap_freq_weights': [(0, 0.1004),
                       (0.14, 0.0476),
                       (0.35, 0.0153),
                       (0.56, 0.0040),
                       (0.77, 0.0009),
                       (0.98, 0.0001),
                       (1, 0.00015)]
},

That’s a lot of numbers and brackets, but we can break it down and make some sense of it:

• filename points to a text file named with a group of familiar sounding words
• name is set to 'fourteen', the numeric title of the poem we saw at the top of each example version above
• immutable_id is just the numerical representation of name in this case
• distance_weights looks like a bunch of number pairs, where neat integer values correspond to long decimal numbers, and there seems to be a particularly strong weight on the index 1.
• mutable_chance refers to just one number, which looks like it might be a simple probability value (where 0 means “never” and 1 means “always”)
• position_weight seems to refer to some kind of gravity about the poem’s position (whatever that means)
• x_gap_freq_weights looks short for “x gap frequency weights” - maybe it has something to do with the gaps along the horizontal (x) axis between words we saw in the examples?

Let’s go through these and take a look in some more detail.

pen, paper, and wine

The first field refers to a .txt file containing the source text for this poem. If we open it up and read it, we find some words which actually make some sense and generally form syntactically valid English.

This is a stream of series of stream-of-consciousness words I wrote a while ago having drunk a little too much wine, and later typed up and saved on my computer. You may notice that the words have a lot in common with all of the above realizations. In fact, every single word that appears in those versions, and every possible realization of this poem will contain only the words above.

So how exactly do these words get jumbled into what we see in the generated realizations of the poem? Many word sequences in the realizations perfectly align with excerpts from the source text, but a whole lot more don’t. The answer lies in our second field of probabilities, distance_weights, but before we can dive into that we need to revisit elementary school for a minute.

fourth grade

Let’s talk math for a second – it’ll be fun, I promise.

We’re all very familiar with random processes. We use them all the time – shuffling a deck of cards, flipping a coin, throwing dice, and so on. When we flip a coin, we have a 50/50 chance of getting heads or tails. When we roll a die, there is a 1/6 chance of any given number landing on top.

There’s a lot of regularity to these systems. In fact, they’re perfectly regular – each discreet possible outcome has exactly the same probability to occur as every other. That’s fine and all, but not extremely useful for the purposes of making poems. If we used simple dice-throwing to choose completely random sequences of words, we get just that - random sequences of words, and our brains are smart enough to realize what’s going on and are quick to gloss over the contents as random noise, much like how random static on a broken speaker doesn’t sound like there’s a lot going on.

Luckily for us, there are a whole lot of other ways to make random numbers. Say we fill a bag with colored marbles: 5 red, 4 blue, 2 yellow³, and pull one out at random. If we take the number of marbles of each color and divide them by the total number we get the probability of pulling each color out. Each probability is different. We can draw it out like before:

John Cage and state machines

Now things are getting interesting. Say we have a bunch of index cards with words on them, and many of the words are the same. Our words are going to be taken from this beautiful quote from John Cage: “I have nothing to say and I am saying it and that is poetry”. Remember, we don’t have just one card for each word; we have many repeated cards in different amounts. Our word distribution looks like this:

If we were to pick words out of this bag, one after another (and place each back in when we’re done) the resulting sequence might look something like this.

is have and nothing have it poetry is say poetry poetry and have is have and and is have poetry say it am am have and have am say and poetry have and have am is say poetry is poetry poetry have poetry nothing have poetry saying nothing and say

What we just did was take a collection of words and repeatedly choose them at random in a sequence. Because some words are more likely than others to be chosen, certain tendencies begin to emerge in the output of our generative process. This is already becoming very useful for the purposes of making poems with computers, but we can take it further – we’re almost there!

Something we might want to do is have more than just one bag of words that we repeatedly draw from. Instead of having one group of words with varying likelihoods for the entire system, what if every word had its own outcome system?

Let’s define a really simple system with just three words, again from the John Cage quote: “that is poetry”, and choose some probabilities for each word that say what the next word might be:

This is hard to visualize, so lets make it a little easier to read by drawing it out as a graph where each word is a node on the graph and the probabilities are arrows pointing from word to word:

Instead of just randomly picking words out one at a time like we did before, we walk along this graph using the following process:

1. Start at our favorite node (word)
2. Follow one of the outbound arrows to the next node, picking one randomly based on the weights of each available choice.
3. Go back to step 2 and repeat however long we want.

In pseudo-Python this might look something along these lines:

def generate_words(graph, word_count):
    current_node = pick_random_node_on_graph(graph)
    output_words = []
    for i in range(word_count):
        output_words.append(current_node.word)
        current_node = pick_next_node(current_node)
    return output_words

What we’ve just built is a fantastic tool known as a Markov chain. It’s a way of abstracting sequences of events as probabilities pointing between states in a graph. Despite their conceptual simplicity, Markov chains can be used to model highly complex systems with often surprising behavior.

from sentences to graphs

So how do we go from a bunch of wine-drunk text to a Markov chain? If we wanted to model our text very accurately, we could follow a process something like this:

1. Break our source text into individual words
2. Pick the first word
3. Add the word to our graph (if the word is already in the graph, we just move to that node and continue with the next steps from there instead)
4. Look at the next word in the text and add a node for that in the graph (if it doesn’t exist already)
5. Add an arrow pointing from the word we’re on to the word that follows it. We give the arrow a weight of 1, and if our current word already has an arrow pointing to that word, we add 1 to the existing weight.
6. Continue to the next word in the text, jump to step 3, and repeat for the entire source text.

Again, we can translate this to some mostly-working Python for a little more precision:

def build_poem_robot(source):
    nodes = []
    links = []
    words = source.split(' ')
    for i, word in enumerate(words):
        if word not in nodes:
            nodes.append(word)
        for link in links:
            if link.source == word and link.target == target:
                link.weight += 1
                break
        else:
            links.append(Link(source=word, target=words[i + 1], weight=1))
    return nodes, links

This process will result in a very accurate modeling of the statistical tendencies of our source text. If you want to imitate a body of text like a politician’s tweeting style accurately, this is a great way to do that. But for the purposes of making poems, modeling our source text accurately may not be much of a concern. What if we want our text to go backwards, or jump forward skipping every other word, or slip in and out of coherent sequences and chaotic nonsense?

doing things wrong

Enter distance_weights. By assigning a weight to every relative position, we can very compactly encode an enormous amount of behavior into our Markov chain. As a reminder, our weights from before look like this:⁴

'distance_weights': {-3: 14.1,
                     -2: 17.2,
                      0: 19.8,
                      1: 168.7,
                      2: 19.5,
                      3: 17.2,
                      4: 22.5,
                      6: 19.4,
                     10: 13.5,
                     16: 6.5},

We can use this information in our above process (algorithm) for deriving Markov chains from text simply by injecting it into step 5. Instead of just adding a weight of 1 for the following word in the text, we add a weight of 14.1 for the word 3 words behind, then a weight of 17.2 for the word 2 behind, and so on until we add a weight of 6.5 to the word 16 spots forward. The number on the left hand side of the colon represents the distance from the current word, and the number on the right side represents the weight for that word. Here’s how this translates to code:

def build_poem_robot(source):
    distance_weights = { ... }
    nodes = []
    links = []
    words = source.split(' ')
    for i, word in enumerate(words):
        if word not in nodes:
            nodes.append(word)
        # Cool stuff starts here
        for key, weight in distance_weights.items():
            target = words[i + key]  # Ignoring IndexError possibility...
            for link in links:
                if link.source == word and link.target == target:
                    link.weight += weight
                    break
            else:
                links.append(Link(word, target, weight))
    return nodes, links

This process of using a series of distance weights instead of just one for every word means we end up with really complicated graphs in the end. When we run it on our original wine-drunken source text, we get something like this:⁵

And that’s exactly what’s happening on we accidentally imagine — 32 source texts, each with their own unique set of distance_weights, are converted into unique Markov chains which are used to generate 32 poems.

odds and ends

We’ve gone into a pretty large amount of detail for how we build the words of our poems, but several elements of the generative process haven’t been touched on. Luckily, in our construction of a Markov chain, we’ve already covered the foundation necessary to understand everything else going on:

• Many poems often don’t go through the Markov process, and instead appear word-for-word as they do in their source text. With every generation of each poem we roll for a chance to use the Markov system using the probability defined in that poem’s mutable_chance field.
• The number of words in each poem is determined by rolling a weighted random number on a predefined set of weights shared by every poem.
• Visual gaps between words (also causing indentation) are randomly inserted according to a frequency determined at the beginning of each poem generation by rolling on the x_gap_freq_weights set of weights.
• The order of the poems on the page changes as well. We control this by taking the position_weight field of every poem and repeatedly roll on those weights, placing each poem into the document as they are picked.
• Vertical gaps between poems are determined by another set of weights shared by every poem.
• The weights themselves that form all of these probabilities are mix between being hand-picked with Intentions and being randomly generated themselves.

bringing it all together

When you visit http://weaccidentallyimagine.com, your computer tells a server to give it a webpage. The server then runs a program that uses all of these processes to spin together 32 poems, transform them into an HTML document, and send them back to you.⁶ Every time you visit or refresh the page a completely new set of poems is generated just for you.⁷

So anyways, why go through all this trouble to make a bunch of gibberish? I’ve already written too much, and later I’ll probably follow up with more on the why’s behind this all, but in brief for now: I don’t care about details, and this allows me to avoid writing them in the first place. I think the word should is inherently violent, so I’m trying to move away from saying what the things I make should be. I want the things I make to help bring people together, so I make things which fundamentally have more than one side to them.

At least, that’s the idea.