Could artists hand-drawn designs be converted to 3d… Automatically?
New Computer Graphics Technology Idea
The artist draws a complete still of a whole scene from several (4-8) angles and then the computer automatically models a virtual 3d version of that scene.
The user would then tell the computer what each automatically generated 3d model and/or part in the scene represents and the computer would then calculate how its physics would work and how it would animate.
The user would then simply modify these scenes via their elements or through camera movements, etc… to create frames over time which would result in animation when not using motion capture technologies or models (either full scale or scaled).
Finally, since the original scenes would be drawn either with traditional media like pen ink, gouache paint and so on, or with something like photoshop with a certain colour palette brush or pen style etc… the computer would automatically calculate the rendering for the final image to match what the artist’s chosen media looks like in his or her finished artwork. This would include things like the lighting, colours, lines, shapes, perhaps textures and so on.
How Does That Differ From Current Technology?
The difference between the above idea and current computer animation media is that the artist would not have to model the elements in the scene themselves, instead they would draw in the same way as in traditional animation, just from different angles and the computer would automatically do the 3d modelling for the user, instead of such drawings only being used as visual reference while the artist is forced to model it in 3D from scratch.
The artist would not have to set up the physics, bones for character movement and so on either, they would simply identify what each element in the scene is and the computer would automatically set up the physics based on the artists’s chosen options on how things should animate.
Finally, the rendering system, instead of being based completely on algorithms by themselves to achieve a certain effect, like interpolation, it would be on a case per case basis, changing depending on the current frame’s composition rather than the virtual 3D calculations alone; based completely on the artist’s design and intended style with the use of things like colour palette, contrast, line and so on; a visual scripting system or equivalent could be used to connect the ins and outs of those different aspects of the design to how they will be rendered on a dynamic or conditional basis, like what is done with custom scripting for games or AAA animation as built-in user-friendly features.
Why this instead of current media?
The reason to use animation over live action is to more precisely and directly achieve the directors vision by being able to have the scenes, characters and look without the use of real sets, actors, lighting and so on.
Traditional animation has a good visual impact when watching it, but is inflexible when it comes to doing very dynamic animation, although it is capable of doing so, this is usually somewhat limited because of the consumption of time in doing so.
Computer animation is very flexible at making dynamic scenes without consuming too much time. However, because of the very indirect nature of the artists having to build everything in a virtual environment from the ground up, rather than simply drawing a scene, the impact in comparison to traditional animation becomes generally weaker.
Finally, although with many advanced and continually developing technologies for rendering animation making it possible to make extremely attractive, realistic and impressive computer animation, the result oftentimes can still look synthetic due to the nature of most of these technologies (excluding ones based on video footage for example) being that they are rendering scenes based on algorithms making the resulting image lacking the depth found in a hand rendered image.
One more note
In order to avoid the artificiality inherent in interpolated animation, motion capture technologies and classic hollywood special effects such as full scale or scaled models either filmed (using the artists hand drawn art as a reference for the final rendering of the production’s image) or animated via stop motion should also be employed. How much these methods should be employed should depend on such a method’s appropriateness for the given result.
Advantages over other current media if the above medium could be developed
- More impact than usual computer animation (animation in games like Guilty Gear Xrd are an exception)
- Generally a lower quantity of artwork required to be hand drawn than traditional animation.
- Dynamic scenes less time consuming (like computer animation)
- Casting is less difficult, since it becomes purely voice casting and real sets and shooting on location would not be necessary (like most animation)
- Setting up animation less difficult than what I think it is in current computer animation in terms of setting up the physics, bone systems, weighting etc…
- Easier to do scenes involving supernatural elements, scenes, representations etc… than live action (like most animation).
- It would be very costly to develop such software
- Not only the development but the polishing and refining of such technology would be a massive undertaking
- It is a different direction than the current big studios like Pixar’s focus, being more on specific improvements on physics simulation for example so it would be hard to achieve funding soon
- Even if this could be achieved, it may be difficult, like much other automatically computer-generated technology, to consistently get pleasing results
- The artist may be forced to limit or standardize their artwork specifically to be better scanned and converted by the technology
- Brand new software and standards would introduce a brand new learning curve to master on top of everything else a modern animator needs to know.
Such a medium would have the potential to at least look as good as traditional animation in general while being generally more engaging with greater ease in producing dynamic camera movements, and more easily produce accurate animated perspective than in traditional animation.
Such a medium would be able to generally be as engaging as computer animation while having a potential to look generally better with its rendering being based on on hand made visual art (it could even be oil painting for example) rather than solely algorithms.
Such a medium, with the use of motion capture technologies, stop-motion and models (replicas, real models or scaled) would allow for realism of movement that could match live-action while being simultaneously more easily able to produce scenes which are generally more difficult to produce outside of animation.
Not Already Out There?
The closest thing I can think of to doing this already is via amalgamating a combination of resources and production strategies. For instance employing stock 3D models as is or edited along with one’s own custom models. Thus not everything has to be done from scratch, yet the director can still achieve his desired results. The only issue with that is licensing, especially for games where it can get quite costly to employ the assets of numerous artists while not being exactly of one’s own design from start to finish. The not-so-distant future of animation media I am predicting then would break that compromise opening new exciting possibilities for animation, movies and games, like never before.