Latent Sheep Dreaming
Machine for Extrapolated Visual Inception
Jiewen Wang, Ollie Hsieh, Jack Gale, Sienna Sun, Sang-won Leigh
Drawing inspiration from Do Androids Dream of Electric Sheep? Latent Sheep Dreaming is an installation exploring tension between human and machine agents in generative processes. Taking place in an empty room, a participant walks up to a stylus console and is instructed to draw sheep. Stylus input is transformed into a matrix of locally and globally affected outputs, increasingly unrecognizable with system influence and connoting of how an artist relinquishes creative ownership to machines. The output is then projected onto a wall and physicalized by a pen plotter.
This project was a joint collaboration between IPDL and Machine Poetics group. It was accepted to DIS 2020 and can be seen here.
Non-digital components in Latent Sheep Dreaming
In addition to the digital system, the installation comprises an AxiDraw v3, Wacom PTH451 Intuos Pro Pen and Touch Tablet, and a Benq Mw632st projector.
Components: AxiDraw v3, Wacom PTH451 Intuos Pro Pen and Touch Tablet, Benq Mw632st projector
Procedural design model
One of two components of the digital generative system, the rule-based system allows for definition of a transformation rule to be applied globally or group of points, with rule types ranging from insertion, translation, segmentation, and interpolation. Shown below is the application of a stacked set of transformations.
Stylus input (left), rule-based system output (right)
VAE model
The VAE model is a two-layer LSTM network, wherein the first is bidirectional and compresses the sketch sequence into a hidden state and encodes it as the latent variable z. The second is the decoder, a unidirectional LSTM followed by a Gaussian mixture model that generates new sequences on the latent variable. This model enables style transfer from arbitrary inputs to the operator’s input and displacement parameters can be tuned to control degree of influence.
VAE model schematic diagram
The end to end system
Stylus input is in the system, with varying degrees of influence from the human operator. The output is projected onto a screen and transcribed with the AxiDraw.
System output projected by Benq Mw632st projector