I was obsessed by this idea of bridging the gap between text and image fully, translating pieces of writing into abstract art that somehow conveys the fundamental voice of the piece.
But first, I needed to create a set of rules/logic for how text indicators translates into visual components.
I kicked off this exploration by hosting a series of “translation experiments.” I asked a few individuals to read a short prose piece and then respond by doing the following:
(1) Write the word from the piece that best describes it
(2) Choose the color that best represents the piece
(3) Make a composition that visualizes the text using 15 pre-assigned shapes
I was fascinated by the hugely divergent responses and compositions that were being made in response to the same piece of fiction. Ultimately, I decided that there was no “right” answers here. My algorithm would be a highly subjective expression of my creativity.
Some of my early explorations involved playing with different variations and permutations of the mappings I outlined above. Below is the output of the same five texts through three different versions of my algorithm.
The five texts I chose are:
1 A love poem
2 A scary story
3 One Fish, Two Fish, Red Fish, Blue Fish
4 A political news article
5 A short story called “Paper Menagerie”
This algorithm generates a colorful collage of circles. Each circle represents a word, and each cluster of circles represents a sentence. The size of each circle is determined by word length, and the number of circles in a cluster is determined by sentence length. The distance between different words in a singular cluster is determined by lexical diversity. The color of each circle is determined by the sentiment score of the sentence, while the background color adjusts to the total average sentiment.
The love poem tends to generate pale irredescent hues. The sentences are shorter in length in the love poem, thus generating smaller, less dramatic sentence clusters. The lexical density produces fairly irregular circle clusters.
The horror story consistently creates large dark circles intermixed with occasionally brighter colors. There are many long run-on sentences in this text, so the circle clusters are large and overlapped with many layers.
One Fish, Two Fish, Red Fish, Blue Fish
This algorithm consistently outputs mild gray-tone hues due to the primarily neutral score of the text. The circles are also smaller and more spaced apart due to the prevalence of short words and the lexical diversity of the text.
In comparison to the Dr. Seuss book, this article used much larger words and has larger circles. The sentiment is highly mixed, though it skews slightly positive. Therefore, the colors reflect a mix of both dark and light hues.
Longer prose pieces in general tends to have reduced lexical diversity and create more uniform, concentric visual output. The story is bittersweet and has mixed sentiment scores, resulting in an eclectic color range.
This algorithm relies on part-of-speech to create a continuous network of shapes. Small circles represent nouns, ovals represent adjectives, lines represent prepositions/conjunctions, and polygons represent verbs. This gives an accurate read on a text’s lexical density, or the different ratios and quantities of lexical items in a text. More lexically diverse compositions are larger while less lexically diverse compositions appear smaller and more compressed.
Generally, there are more adjectives, nouns, and prepositions/conjunctions in relation to action words like verbs. Therefore, there is more empty space.
Sharp polygons are more frequent and noticeable due to the higher number of action words. This piece is linked together more closely than the love poem. So we can assume it has a smalller lexical diversity.
One Fish, Two Fish, Red Fish, Blue Fish
The composition is composed of more empty spaces and fewer shapes because of the piece’s shorter length and simper content. There are lots of adjectives and objects, resulting in more ovalesque and circular shapes.
This composition is a lot more complex and busy with a wide variety of densely compacted forms. The larger ovals are the result of larger magnitude levels (and thus larger levels of lexical diversity).
The lexical diversity of the short story is smaller, resulting in more condensed clustering. The predominant geometric forms here are the circular and ovalesque shapes because the story is highly character-driven.
High levels of emotionality translate into color. Positive language translates into round shapes, and negative language translates into sharp angles. Neutral sentences become crosshatches. All of these compositional rules operate under an umbrella of randomness, transforming a single input into an infinite source of output.
Each of the three videos is a chronology of 10-12 iterations of the algorithm, selected from hundreds of iterations. While each clip in itself is a miniature portrait of the piece, I was interested in what happened when these clips were strung together and sequenced intentionally, creating a new narrative, one that surpassed the sum of its parts.
I created a poster for each prose piece which compiled together 100 iterations of the same piece through my algorithm. Within each poster, I sought to convey a unified visual language that girded together these iterations. I was interested in how a collection of individual compositions can become a textile, or an alphabet of shared forms.