How Research Arts rewired its workflow

How did the brief or purpose of the project evolve from conception to completion?

Initially I wanted to be able to develop some more efficient rigging techniques for video game development and animation. Over the last few months, a lot of my focus has been on creating toolsets that allow me to focus more rapidly with creative development, being able to spend more time designing things with pen to paper, then using custom pieces of software with LLM's to flesh those things out and add incidental detail/create happy accidents.

I've always been interested in curating automated processes and building generative structures that yield surprising outputs. 

What first inspired you to explore the relationship between AI, CGI and artistic control?

I've been experimenting with AI since 2019. I've always been interested in curating automated processes and building generative structures that yield surprising outputs. To me, it felt like quite an alien way of getting surprising outcomes. My early work mainly involved training GAN's with incoherent datasets, using the output to drive other elements of CG, so shaders, height maps things like this. 

As the field has matured and become wider in its capability this process naturally extended to 3D software, as well as connecting pieces in the post pipeline that aren't conventionally connected.

Can you talk us through the creative process behind blending AI with traditional CGI techniques?

A straightforward example is you may create a series of renders using abstracted 3D forms you author by hand, then you may combine them with archival works from your library. These create surprising forms that are essentially the average of a series of images that are completely unrelated. 

This allows you to have control over the process but to foster a culture of accidents and exploration within the automation.

This form can then be extruded using img->3D models, and then constructed and rigged together like building a real robot. Of course, these systems create imprecise meshes so what you then do is create bits of software that can add precise joints, swivels and things like this to connect them. This allows you to have control over the process but to foster a culture of accidents and exploration within the automation.

Did you set any specific rules for where AI would be used, versus more traditional techniques and human craft?

I never use generative ai to create a final image, for instance something like using Midjourney or Nano Banana. I think it takes all the fun out of it. For me the tools are far more interesting when pushed to breaking point or used in a way that creates a structure that surprises the user in the same way that say for instance flinging paint at a canvas does. However with CGI pipelines you need precision and control, so using it to build tools is very different than using it end to end to create an image or a video.

I never use generative ai to create a final image, for instance something like using Midjourney or Nano Banana. I think it takes all the fun out of it. 

Which areas of the creative process do you feel are most important to keep human-led, such as ideation or finishing touches, or does that vary from project to project?

The direction, character design, locations, machines, whatever is in the scene – its important to me, at least, that these are authored by human intention rather than text prompt generative ai. Where it can be useful is things like say for instance, you have a rendered image, you want to add some scuffs or organic detailing to some metal, it can be very good at doing that. Or say for instance if you have a simple light shift, say lights on to off, some video models can take in two rendered stills and interpolate between them, saving rendering time. It's very useful for things like this.

It's expanded whats possible and the level of detail thats achievable; I think the way neural networks can add organic detail is amazing.