Pixar and inverse kinematics
I recently came across Spacetime Constraints by Witkin and Kass, a 1988 paper proposing a new methodology for how animators should map out character movement.
Instead of an animator drawing every single keyframe of an animation, which takes too much time, why don't we just use our models of physics to draw it for us?
With this approach, the animator specifies what the actor has to do in only a few keyframes (e.g. drawing the starting and ending positions of a jump) and how the character should do it (e.g. "not wasting energy").
Using just Newton's laws and the computers of the time, we form and solve a large, but very sparse, constrained optimisation problem (the details of which we will skip over), and out of this we obtain the actor's position through space and time and can map out the full motion.
The following is an informal generalisation of what is going on here and how I think this approach from the '80s could map onto the modern engineering process.
- We start with a dynamical system (the character and environment)
The character has an infinite set of possible states within its physical environment and we seek to solve for a specific state trajectory over time.
- A user describes the solution with a set of partial datapoints**
These provide constrain what the solution can be and should be information-dense,
meaning the user should describe what they want, and nothing more.
e.g. in this case, it's better for the user to draw exact keyframes rather than describe the general movement in text.
- We use a built-in prior to compute the result
We use a built-in prior on the behaviour of the system (e.g. by enforcing physical laws) to further reduce the set of solutions down to a unique solution or at least a manageable set of solutions. When animating character movement, the priors are Newton's Laws.
For thinking about the inverse design of physical objects and tools, Number 2 is the first problem to solve:
How can users describe the physical behaviour of the tool they want, and nothing more?
With programs, we can define the intended behaviour of the system formally in terms of pre/post condition contracts or temporal logic. Or informally, with a natural language specification, or just with input/output examples (which is essentially the same approach as in the paper).
It seems like there are many ways to describe the results we want at varying levels of abstraction. Going back to animation: maybe, the least abstract (and most specific) option would be to to define each keyframe of the motion we want.
And the most abstract might be to describe the motion in a single word. Like "jump".
Neither the animator or the viewer actually care about what each individual keyframe looks like and it would be a pain to draw each one out... but they do care enough to say not every "jump" animation is appropriate.
Using logic:
Let Ideal Animation := I, "jump" := J, Specific Animation := S
S < I < J
So, roughly speaking: we don't have to manually define all valid sets of keyframes.
Instead, our interface should work at a more abstract level that is just specific enough for us to define the result we want from the system.
Specific Animations < User Interface < Text Description
If we consider Animations as a poset category with sets of animations as objects and morphisms as inclusions, then we can define an idempotent monad as a projection from this category into a more abstract interface, represented in this category.
How can we define this for physical open systems with a concrete example ?
And.. is there a direct relation with different levels of abstraction within dynamical systems?
E.g. modelling intput-output behaviour of tool vs modelling input output behaviour of tool+environment
And can we make this formal?