Text-to-Video Experiments // BrXnd Dispatch vol. 016.5
A fun guest post with some experiments with new tools.
You’re getting this email as a subscriber to the BrXnd Dispatch, a (roughly) bi-weekly email at the intersection of brands and AI. On May 16th, we’re holding the BrXnd Marketing X AI Conference. Tickets have gone much faster than expected, and the waitlist is now open. I fully expect to release more tickets, but until I can better grasp all the sponsor and speaker requirements, it’s safest to do it this way. Add your name to the waitlist to be immediately informed when more tickets are available. With that said, sponsor tickets are still available, so if you’d like to sponsor the event, please be in touch.
Today I’ve got a guest post for you. Sean MacPhedran of SCS (also my partner on the Ad-Venture game) has been playing with some text-to-video stuff and was kind enough to do some BrXnd experiments and a write-up. I’ll let Sean take it away. - Noah (PS - this came out of a conversation on BrXnd Discord. If you want to have conversations like this, please join us there.) - Noah
Text-to-Video is going to be big. Bigger than Text-to-Image, I think. Right now, the tools are reminiscent of the early image generation days with apps like StarryAI and Wombo Dream—video is about a year behind images. But Dentsu just released research that about 27% of the US online population has already used some type of Generative AI tool, so it’s likely that when everyone can create videos on a whim, it will be explosive.
Over the Christmas holidays, I got hooked on making simple (and unavoidably weird) videos with Deforum Stable Diffusion, a precursor to tools like Runway’s Gen 1 and Gen 2 text-to-video platforms. With it, you can create pure text-to-video without a reference image and transform existing videos. Figuring out how to use it and generating one video took about 16 hours (it was a very rough, AI-weird trailer for an imaginary Hangover 4 movie called Van Gogh’s To Vegas). For those who tinker, my main challenge with the Deforum Colab is that the user generates multiple images from Stable Diffusion in one pass. If you are trying to create a narrative—as I was—it generates around 20 images based on 20 prompts simultaneously before interpolating them into one another. Take them or leave them, and regenerate all of them again. Continuing this exploration of text-to-video, we used Kaiber.ai to add an animation to an AI video for SCS we created mainly with ControlNet, which runs a simplified (and very limited) interface for Deforum.
This brings us to a fun new discovery, which is an elegant way to bring logos to life with Kaiber. Tinkering the other night, I tried to transform logos into their experiences with the platform. After a while, the results were nice. I picked a few favorite brands—Corona, Spotify, LEGO, and NYC—and noodled on prompts that would evoke their spirit. Again, the results were pretty fantastic. If you’d like to try it, I recommend dropping the target logo onto a 1:1 size JPEG, uploading it, and just playing with prompts until you get an interesting result. Have fun!
Here’s the original I posted on LinkedIn:
BRXND SPONSORS
A huge thank you to the many supportive sponsors who are making the event and my work possible. If you would like to sponsor, please be in touch.
It’s Noah again. Sean was also kind enough to make a BrXnd version:
See you in May!
— Noah
PS - If you haven’t joined the BrXnd.ai Discord yet and want to talk with other marketers about AI, come on over.