Adobe's next big project is an AI that can upscale low-res video to 8x its original quality

Date:
Wed, 24 Apr 2024 20:03:26 +0000

Description:
Blurry clips improved by Adobe's VideoGigaGAN AI can display hi-res textures and more making videos look realistic.

FULL STORY ======================================================================

A group of Adobe researchers recently published a paper on a new generative
AI model called VideoGigaGAN and we believe it may launch on a future
product. What it does is upscale low-quality videos by up to eight times
their original resolution without sacrificing stability or important aspects of the source material. Several demo clips can be found on the projects website showing off its abilities. It can turn a blurry 128x128 pixel resolution video of a waterfall into footage running at a resolution of 1,024x1,024 pixels. Post by @luokai View on Threads

Whats noteworthy about the AI is it doesnt skimp out on the finer details. Skin texture, wrinkles, strands of hair, and more are visible on the faces of human subjects. The other demos also feature a similar level of quality. You can better make out a swan swimming in a pond and the blossom on a tree
thanks to this tech. It may seem bizarre to be focusing so much on skin wrinkles or feathers. However, it is this level of detail that companies like Adobe must nail down if they aim to implement image-enhancing AI on a wide scale. Improving AI

You probably have a couple of questions about the platforms latest project like how does it work? Well, its complicated.

The GAN in VideoGigaGAN stands for generative adversarial network, a type of AI capable of creating realistic images. Adobes version is specifically based on GigaGAN which specializes in upscaling generated content as well as real photos. The problem with this tech, as TheVerge points out, is that it cant improve the quality of videos without having multiple problems crop up like weird artifacts. To solve this issue, Adobe researchers used a variety of techniques.

The research paper explains the whole process. You can read it yourself to
get the full picture although it is dense material. Basically, they
introduced a flow-guided propagation module to ensure consistency among a videos frames, anti-aliasing to reduce artifacts, and a high-frequency
feature shuttle to make up for sudden drops in detail. There is more to VideoGigaGAN than what we just described, but thats the gist of it. Potential inclusion

Will we see this on an upcoming Adobe product or roll out as a standalone
app? Most likely at least we think so.

In the past year, the company has been focusing heavily on implementing artificial intelligence into its software from the launch of Firefly to Acrobats new assistant . A few months ago during Adobe MAX 2023, a video upscaler referred to as Project Res Up was previewed at the event and its performance resembles what we see in the VideoGigaGAN demos. An old movie
from the 1940s goes from running at a 480 x 360 image resolution to a crisp 1,280 x 960. Blurry footage of an elephant in a river becomes crystal clear. The presenter even mentions how the software can upscale a clip to four times the original quality.

Admittedly, this is conjecture, but its entirely possible VideoGigaGAN may be the engine behind Res-Up. Adobes future product could give people a way to upscale old family videos or low-quality footage into the movie we envision
in our minds. Perhaps, the recent preview is a hint at an imminent release.

VideoGigaGAN is still deep in development so its unknown when or if itll come out. There are several obstacles in the way. The AI cant properly process videos beyond 200 frames or render small objects, but we'll definitely be keeping an eye on it.

In the meantime, check out TechRadar's list of the best AI image upscalers
for 2024 . You might also like Adobe Firefly's latest image model looks like
a generational leap and I can't wait to try it OpenAIs new Sora video is an FPV drone ride through the strangest TED Talk youve ever seenand I need to
lie down Watch this: Adobe shows how OpenAI's Sora will change Premiere Pro and video editing forever



======================================================================
Link to news story: https://www.techradar.com/computing/artificial-intelligence/adobes-next-big-pr oject-is-an-ai-that-can-upscale-low-res-video-to-8x-its-original-quality


--- Mystic BBS v1.12 A47 (Linux/64)
* Origin: tqwNet Technology News (1337:1/100)