.gif)
had bodies?
What if words
I designed an algorithm using Python and sentiment analysis tools to translate text input into visual output. I also planned an installation showcasing my work.
Tools
↗ Python
↗ NLTK
↗ JS/HTML
Skills
↗ Generative coding
↗ Natural language processing
↗ Video art
Deliverable
An overhauled Fonts panel experience with new hover states & microinteractions to clarify how to successfully navigate and configure typographic settings.
The Ask
Understand how users work with the fonts panel, identify areas to optimize, and redesign the end-to-end experience of managing fonts on a site.
Origins
As a designer and writer, I've always been interested in pushing and challenging the lines between text and image. This was a concept I at first explored through typography, learning about how text can become far more than the sum of its words, become an expressive visual object.
I ended up doing a lot of work over the course of my time at Yale that was focused on rendering type into texture. For example, I experimented with creating typefaces and blending the line between type and image.
Shadow type - rendering type into embodied forms
Translation experiments
I was obsessed by this idea of bridging the gap between text and image fully, translating pieces of writing into abstract art that somehow conveys the fundamental voice of the piece.
But first, I needed to create a set of rules/logic for how text indicators translates into visual components.
I kicked off this exploration by hosting a series of “translation experiments.” I asked a few individuals to read a short prose piece and then respond by doing the following:
(1) Write the word from the piece that best describes it
(2) Choose the color that best represents the piece
(3) Make a composition that visualizes the text using 15 pre-assigned shapes
I was fascinated by the hugely divergent responses and compositions that were being made in response to the same piece of fiction. Ultimately, I decided that there was no “right” answers here. My algorithm would be a highly subjective expression of my creativity.
Algorithm hypotheses
I began a long journey of experimenting with visualizing textual determinants. I decided to incorporate the following into my algorithm:
Early algorithms
Some of my early explorations involved playing with different variations and permutations of the mappings I outlined above. Below is the output of the same five texts through three different versions of my algorithm.
The five texts I chose are:
1 A love poem
2 A scary story
3 One Fish, Two Fish, Red Fish, Blue Fish
4 A political news article
5 A short story called “Paper Menagerie”
Algorithm 1
Sentiment Circles
This algorithm generates a colorful collage of circles. Each circle represents a word, and each cluster of circles represents a sentence. The size of each circle is determined by word length, and the number of circles in a cluster is determined by sentence length. The distance between different words in a singular cluster is determined by lexical diversity. The color of each circle is determined by the sentiment score of the sentence, while the background color adjusts to the total average sentiment.
Love Poem
The love poem tends to generate pale irredescent hues. The sentences are shorter in length in the love poem, thus generating smaller, less dramatic sentence clusters. The lexical density produces fairly irregular circle clusters.
Scary Story
The horror story consistently creates large dark circles intermixed with occasionally brighter colors. There are many long run-on sentences in this text, so the circle clusters are large and overlapped with many layers.
One Fish, Two Fish, Red Fish, Blue Fish
This algorithm consistently outputs mild gray-tone hues due to the primarily neutral score of the text. The circles are also smaller and more spaced apart due to the prevalence of short words and the lexical diversity of the text.
News Article
In comparison to the Dr. Seuss book, this article used much larger words and has larger circles. The sentiment is highly mixed, though it skews slightly positive. Therefore, the colors reflect a mix of both dark and light hues.
Paper Menagerie
Longer prose pieces in general tends to have reduced lexical diversity and create more uniform, concentric visual output. The story is bittersweet and has mixed sentiment scores, resulting in an eclectic color range.
Algorithm 2
Speech Network
This algorithm relies on part-of-speech to create a continuous network of shapes. Small circles represent nouns, ovals represent adjectives, lines represent prepositions/conjunctions, and polygons represent verbs. This gives an accurate read on a text’s lexical density, or the different ratios and quantities of lexical items in a text. More lexically diverse compositions are larger while less lexically diverse compositions appear smaller and more compressed.
Love Poem
Generally, there are more adjectives, nouns, and prepositions/conjunctions in relation to action words like verbs. Therefore, there is more empty space.
Scary Story
Sharp polygons are more frequent and noticeable due to the higher number of action words. This piece is linked together more closely than the love poem. So we can assume it has a smalller lexical diversity.
One Fish, Two Fish, Red Fish, Blue Fish
The composition is composed of more empty spaces and fewer shapes because of the piece’s shorter length and simper content. There are lots of adjectives and objects, resulting in more ovalesque and circular shapes.
News Article
This composition is a lot more complex and busy with a wide variety of densely compacted forms. The larger ovals are the result of larger magnitude levels (and thus larger levels of lexical diversity).
Paper Menagerie
The lexical diversity of the short story is smaller, resulting in more condensed clustering. The predominant geometric forms here are the circular and ovalesque shapes because the story is highly character-driven.
Algorithm 3
Polarization Web
This program creates a word web of polarized content. Each line represents a word, and each line has an origin point that is dependent on the Gunning-Fog Index. The “less readable” a text is, the more variation exists between origin points. Sentiment is visualized by line height and direction. More intensely positive words will reach higher on the canvas and more intensely negative words will reach lower on the canvas.
Love Poem
The love poem expresses exclusively positive sentiment, therefore generating these bright, upwards compositions. The readability is higher; therefore, the origin point is more consistent and more clearly defined.
Scary Story
The scary story is overwhelmingly negative, generating dark output. Because it has lower readability, the origin point is less defined.
One Fish, Two Fish, Red Fish, Blue Fish
Because the words and sentences on average are shorter, the width of the composition is much smaller. The composition is also relatively mild in terms of sentiment, generating more mid-range y-values.
News Article
Words are a bit closer to the center on average due to greater neutrality in the article’s language. The width of the composition is wide, indicating that the article uses larger sentences and words.
Paper Menagerie
The short story is an extremely dense word web. Because of the length and sentiment range of the text, it includes many layers which range from shorter lines to lines that reach the borders of the page.
I didn't want to make an infographic.
Evolving the algorithm
Although these early explorations were a valuable technical exercise, they were still falling short of my goal.
I didn't want to make an infographic. I wanted to create an immersive, highly visual experience that carried the voice of the text into the composition.
I decided to make a few changes to my project:
(1) Animation
Rather than static images, I decided to capture the canvas as the text was being read by my algorithm. This pivot to animation better personified the element of narrative.
(2) Algorithm modifications
I was most drawn to my explorations where shapes began to blend together to create new, abstract forms. I modified my algorithm to prioritize the creation of large, bold, overlapping shapes. I also prioritized color as a variable because I was most drawn to my explorations which made vibrant use of color.
(3) Personal stories
Rather than using these generic texts, I wanted to plug my own writing into the algorithm. In particular I wanted to center my work around individuals who feel “invisible.” I was inspired by the idea of an algorithm translating these invisible voices into stark visual reality.
Installation
For my installation, I focused on three pieces of short prose. I ran each piece through my algorithm hundreds of times, and compiled the output into a single video collage for each prose piece.
Algorithm Design
High levels of emotionality translate into color. Positive language translates into round shapes, and negative language translates into sharp angles. Neutral sentences become crosshatches. All of these compositional rules operate under an umbrella of randomness, transforming a single input into an infinite source of output.
Video Collage
Each of the three videos is a chronology of 10-12 iterations of the algorithm, selected from hundreds of iterations. While each clip in itself is a miniature portrait of the piece, I was interested in what happened when these clips were strung together and sequenced intentionally, creating a new narrative, one that surpassed the sum of its parts.
Tryptich
I created a poster for each prose piece which compiled together 100 iterations of the same piece through my algorithm. Within each poster, I sought to convey a unified visual language that girded together these iterations. I was interested in how a collection of individual compositions can become a textile, or an alphabet of shared forms.
Loved
See full video ↗Beautiful, brilliant, charming. He was beautiful, brilliant, and charming in a way that made people alert, but not suspicious; intrigued, but not overwhelmed; impressed but not bitter. His life was made easy for him. He received kindness he did not deserve. He found himself agreed with before he knew what he believed.
Above all, he wanted to be disgusting. He wanted to make people look down, step back, turn away. A disgusting person is loved more truly than he could ever hope to be loved.
But he was beautiful, brilliant, and charming, and to the very end they adored him.
.png)
Alone
See full video ↗She's been dreaming of pregnancy lately. In her dreams she never knows the father; she doesn't even wonder about him.
It's just her and the thing growing inside of her, which pulls lovingly at her blood vessels and hums songs through her skin.
She's pregnant for years, or maybe a lifetime. Her stomach inflates into a mountaintop. The thing inside her grows wrinkles, but she loves it because it is hers alone. When it is time to say goodbye, she cries miserably.
Then she is awake, lying in the bed she has grown old in. It is lonely, mothering phantoms.
.png)
Strong
See full video ↗The women in my town specialize in erosion. They are soft bodies, slurred skin, hands in cursive. They get smaller every Christmas.
The girls around here are born with mountain voices and buried with dead things in their throats. They peel away into brittle surfaces, like fallen leaves: heard when broken.
Sometimes they clasp hands and unravel. They hum off-key to remind themselves they can make ugly sounds. They yell to remind themselves they are capable of unkindness.
When they are frightened, they tangle their hands in each others’ hair and make neat braids. They remember that they have ten strong fingers, two sturdy feet, a head that thinks. It is not a funny thing, but they cannot help laughing.
.png)
Conclusion
The process of refining this algorithm was extremely grueling, but it was also so much fun! I loved the aspect of chance and the experience of “discovering” beautiful compositions.
I think this algorithm is far from perfect and I’d love to continue exploring this concept. Eventually I’d love to publish this online and allow visitors to insert their own text and make custom prints.