Black and white movies: Graz’ artificial intelligence makes it possible

Countless films are stored in film archives around the world Movies with black and white clips. Since the 1970s, there have been repeated attempts to admit it afterwards coloringBut the effort is great and the colors often don’t match reality. At TU Graz, together with the Graz film restoration company, prof Algorithm based on artificial intelligence TU Graz announced on Monday that it was developed to efficiently and realistically colorize old film slides.

Realistic colors in black and white movies thanks to artificial intelligence

Historical films are often lacking in this particular thing: color. With artificial intelligence, old recordings should be able to be seen in a new color – in realistic colors, it has been confirmed. The central requirement is met by using self-learning neural networks. It can be dynamically affected by user interaction. The software application was developed by computer scientists headed by Thomas Bok of the Institute of Computer Graphics and Visualization at TU Graz with the Graz-based company HS-Art, which specializes in the restoration of historical films.

The development that brings color into the past combines deep learning technologies with interactive and automated coloring technologies. The result is an algorithm for a mostly automatic but user-driven inking process. As Bock explained: “You always need someone who knows from historical records what the clothes, facades, etc. looked like at the time. Was the soldier’s uniform green or blue? No algorithm can decide that. But it can learn from it.” He heads the Vision, Learning and Improvement (VLO) group at the University of Technology Graz.

The algorithm needs training

The algorithm must be fed a large set of training patterns to then automatically colorize historical films: “It’s about coloring films as efficiently as possible with as little user input as possible. It can look like a human being can color a frame of film, and then the program takes over after It colorizes the extra frames,” Bock explained.

The researchers analyzed different and in some cases new methods in the field of automated colorization based on artificial intelligence. Then they implemented the most efficient approach with the HS-Art developers in a prototype application and created a set of training samples. Then human directed control was implemented in order to obtain original and appropriate color schemes.

Scratches can also be repaired

With developed algorithms, films can not only be tinted, but also scratches, dust spots, water and other damages can be eliminated. However, this very clean restoration is not always desirable. “Historic recordings and cinema films in general need a certain amount of noise, the so-called ‘film grain’, otherwise they don’t look authentic to the audience. That’s why software can create this noise and add it artificially again after restoration and colorization,” Bock stressed.

The core algorithm has already been published at an international conference, and the source code is freely accessible. However, software based on this is required for effective use. This was developed by project partner HS-Art and is already part of their range of products: the so-called “Diamond Film Colorizer”, for example, was used in ZDFzeit’s “Hitler’s Power” documentary series to color real historical recordings of the creative.