This Disney AR app could be a stealth play for smart coloring books

In our many perambulations through the halls of museums, who hasn’t wanted to give paintings we pass by a little touch-up in a spontaneous act of yellowism? Disney has created an augmented reality app to allow just this, but given the company’s core competencies I can’t help but wonder if the museum use case is just a smokescreen for another — specifically, the smartification of millions of ordinary story books.

The research project is called AR Museum, and, as you might expect, it’s an app meant to be used while you navigate a museum. But it eschews the obvious AR applications adults would think of, like giving you extra context for the painting you point your phone at, and goes straight to “I wonder what Van Gogh would look like with blue skin?”

Paintings are organized into regions that can be recolored with a touch. So in a way, the app is just a fancy paint bucket; indeed, when you use it to recolor those areas, virtual paint drops fly out of your device and strike the canvas. But there’s a little more to it than that.

Recoloring images like this is easy if the image has been created with that in mind — for example, with skin tones on one layer, the sky on another and so on. If it’s a painting, those areas need to be defined in a post-process, but as the researchers point out: “manually creating the layers that are required for a seamless recolorization of complex paintings is a tedious task for an artist.”

Now is normally where I’d say, “but it’s a breeze for a deep learning model.” Unlike pretty much everyone else in the image processing space these days, the researchers did not use machine learning, but rather decomposed the constituent colors of the painting with ordinary weighting methods. (It’s actually a better and simpler solution for this well-bounded problem.)

The process segmented a painting into useful color regions in one minute, a tenth of the time an artist took to do it manually, and did it better for the purposes of the app. This pre-processing could be done for all the paintings in a museum and thus the kids can entertain themselves turning Starry Night into Starry Broad Daylight while you (in another app, probably) browse the timeline of the Impressionist movement.

But here is where I think they are being sneaky.

Why does Disney suddenly care so much about museums? Can you think of anything else with painterly art that kids might enjoy coloring and recoloring but are prohibited therefrom? Maybe something millions and millions of households have sitting around just waiting to have new life breathed into them by the magic of technology?

Yes! All those old story books! Millions of them, on shelves across the globe. Disney could very easily train the app with all the assets needed to recognize and recolor these oft-forgotten treasures. Suddenly the books are given new life and perhaps AR interactions with them are built on that platform, and more extravagant recastings using the technology. Bambi on Mars, The Fox and the Dalmation, etc.

Of course, they lose a few sales as kids fail to color in the physical books themselves, but they gain a dedicated cadre of kids who see AR as a natural method of interacting with Disney properties. At least that’s how it works in this fantasy of mine.

At the very least it’s a neat tool that could very easily occupy a kid’s hands during a trip to the museum. If it’s more than that, as I suspect, it could be a great way to get a few more years’ use into some classic books.

Featured Image: Disney Research