Bret Victor, the man who gave us an a better understanding of how to teach programming in his landmark article Learnable Programming last September, does it again. In his most recent talk he presents a software prototype for interactive dynamic drawing, a tool for scientists to creatively express their ideas in data visualisations, that go far beyond the narrow perspective of Excel bar charts, but also gets rid of blindly manipulating objects with code by introducing parameterised direct manipulation drawing.
(Start at 29:10 if you’re pressed on time, or even at the short summary he gives at 33:15)
Custom data visualisation tools with direct manipulation possibilities can be built by anyone who knows coding in Processing, d3 or similar tools. However, a certain knowledge of coding and programming is needed to build these. Bret Victor shows in a prototype how these tools could be made available to anyone without touching code, by introducing a GUI that allows for drawing and direct manipulation of the objects with data.
The problem he addresses is the incredibly narrow limits of tools like Excel and Illustrator, who are unable to harness the true potential of pictures, and don’t allow us to create our own visual language. Scientists need to be able to express themselves in a more creative way and put their mental images into information graphics more intuitively. Creative mathematicians and physicists hardly ever use mathematical symbols to sketch ideas, but figure things out in pictures, as Alan Kay explains. A great example for that are the Feynman diagrams, which couldn’t have been invented in tools like Excel or even MatLab.
„Every idea can have its own unique sentence.
Every thought can have its own unique picture.“
(Bret Victor, 4:30 in his talk)
So Feynman was inventing by doodling. How could this translate to the digital world?
Victor categorizes the current approaches to create data visualisation into three categories:
Use: Excel, MatLab, etc.
Draw: by hand or in Illustrator, etc.
Code: Processing, d3, Quartz, Max/MSP, Scratch, etc.
The 4th option he introduces is called Dynamic Drawing. It combines all three, and adds some magic: direct manipulation and parameterisation. While code makes it possible to have dynamic behaviour of the drawings, programming is still „blindly manipulating symbols.“ (8:30)
Input > Procedure > Output
The software has three sidebars. The Data sidebar (representing input) allows you to define variables, make calculations with mathematical expressions and import spreadsheets (CSV files). This is very similar to working with spreads in vvvv. The Steps sidebar (representing procedure) then offers basic operations to draw lines, circles and rectangles. Every drawing action creates a new step that can then be parameterised by data. Also, he shows how to establish relationships between geometric objects with snapping. Objects can also be turned into guides, which then act like temporary variables. To iterate through multiple data points, those steps can be looped. The Measurement sidebar finally represents the output, which then can be the input for another image.
Very clever. Are you listening, Adobe? That’s what we want to see.
A starting point
Victor’s demo is of course just a starting point, as he also says. Much like Fortran in the 1950s, which was the first high-level programming language, this ideas hopefully have a revolutionary impact on software and interface design. It will soon be available on github.
In general, I really hope that we gradually move away from using software with a limited set of visualisation functions (Excel, Illustrator, Photoshop) and a fixed structure to a more open, fluid and creative way of working, playing and thinking in pictures.