Teddy Ni, a co-founder of Magic Patterns backed by Y Combinator, shares an experiment: “We taught an AI some Figma designs of a documentation site. It can now generate similar sites using my own custom components on ANY topic I want, ALL within Figma I can also click on any frame and request updates 🤯”. (This reminds me of the feature in Visual Copilot that I wrote about in #136.)

“Discover how you can import all of your existing Jamboard files to fully editable FigJams! From sticky notes to sketches, watch as we demonstrate how to seamlessly transition your Jamboard content into FigJam, where you can harness the full range of FigJam’s interactive capabilities.”

OMG! Figma window is streamed to GPT-4 Vision, which then provides feedback on the fly narrated in the voice of Steve Jobs. Looking for a way to make this a part of our design crits.

Jordan Singer came up with another wild AI experiment — sketch or design anything in Figma or FigJam and turn it into a functional prototype with one click of a button. This demo is a great example — it built all the functionality correctly based just on a few text labels!

The AI is now integrated into FigJam to “help you instantly visualize ideas and plans, suggest best practices, and, of course, automate tedious tasks, so you can focus on the bigger picture.” What started as an experimental widget Jambot is now a first-class part of the product using GPT-4. I used it this week to create a structure for a presentation which was a useful 0 to 0.1 progression. You can give it a try at the playground or check out how Zander Whitehurst uses it to create crazy flow charts.

I shared the teaser in issue #132, but last week Builder.io introduced Visual Copilot, “a completely reimagined version of the Builder Figma-to-code plugin that will save developers 50-80% of the time they spend turning Figma designs into clean code.” The major difference between Visual Copilot and previous design-to-code tools is a specialized AI model that was trained to solve this problem. The features include one-click conversion, automatic responsiveness, extensive framework and library support, customizable code structures, and easy integration with the existing codebase.

One of the most exciting parts of this announcement is still in private beta and targeted at teams with well-maintained design systems. This feature in Visual Copilot uses AI to map reusable components in your Figma file to those in your code repository and generates code using your existing components when you have them. This could be genuinely useful to get the first rough version ready in no time.

I’m still waiting for access to GPT-4 Vision, but examples like this make me so excited about possible use cases! Imagine using LLM for a heuristic evaluation or pairing design and sketching sessions.

Alright, so in the last issue, I wrote that “plugins for code generation in Dev Mode using GPT-4 might provide an even better result.” It didn't take long for a better example! Ben shows a new feature in Sidekick AI (using GPT-4) for dropping a link to a frame in a Figma file to improve the code generation and even fix visual bugs. This looks freaking amazing.

You can now opt out of beta AI features, so your data will not be sent to Figma’s third-party AI vendor. “Figma’s agreement with OpenAI provides that data is not to be used for model training. Data inputted into AI features is sent to OpenAI for processing and generating AI output. Data is temporarily retained in OpenAI’s environment to provide the services, however it is not used for model training.”

This result is based on an exported PNG, so plugins for code generation in Dev Mode using GPT-4 might provide an even better result.

Dylan Field, founder and CEO of Figma, looks at the relationship between designers, developers, and AI, in conversation with a16z’s David George. In the process, he also demoes Jambot, their new AI widget for FigJam. Love this quote from Dylan: “It [AI] will lower the floor for who’s able to participate in the design process, but also raise the ceiling of what you can actually do.”

Miggi causes the “figception” by using Jambot to come up with ideas for making Figma content. Quite amazing to see how it can be used as a tool for thought (or even programming, FWIW).

Amber Bravo sat down with Jambot engineers and a designer to learn what inspired them to make the widget, and why they’re so excited to see ChatGPT go multiplayer. As a power user of Logseq, I loved this bit from Daniel Mejia on where the inspiration came from: “I’ve been a heavy user of these tools called Networked Thought — especially Roam Research and Logseq — which basically allow you to create pages that link between each other, so you can connect, organize, and trace ideas. More recently, I also found this tool called Albus, which adds a visual feel to interacting with AI, and so I thought there should be a way to connect these concepts to create a potentially useful alternative to ChatGPT.”

Jambot is a free widget from Figma to interact with ChatGPT right in FigJam. Use it to create visual mindmaps, take a multi-threaded approach to brainstorming, or generate ideas with teammates and ChatGPT on the same canvas. Quite amazing that it was born during last month’s Maker Week and is already live!

Use Stable Diffusion to create unique AI images for your designs without ever leaving Figma with this text-to-image plugin. Won the “3rd Product of the Day” at ProductHunt on August 1st.

John Maeda shares takeaways from his Design in Tech Report, thoughts on designing with artificial intelligence, and why we should embrace uphill thinking in a world optimized for shortcuts. See also the livestream “Diving deeper: Designing with AI in the future”.

The first-ever demo of a new plugin by Diagram. Can’t wait for it to become a part of the editor!

The first segment of the opening talk of the second day, with Noah Levin and Diagram team discussing how AI will shape our future and work. Continue by watching Generative AI and Creative Arms Race by Ovetta Patrice Sampson from Google, AI and empowering creative careers by Scott Belsky and Brooke Hopper from Adobe, and wrap up with The crescendo of AI in our collective future by Kanjun Qiu and Reid Hoffman.

Noah Levin, VP of Design, unveils Figma’s vision for AI and shares that Figma has acquired Diagram. (It’s fun to look back at all Jordan’s experiments I shared in this newsletter, starting from 2020.) “In short, AI can help us do more — across every part of the product development process — faster. It’s not a feature, but a core capability; more than a product, it’s a platform that can up-level our work to the plane of problem solving — arguably the core pursuit of our craft, and the reason many of us got into design and product building in the first place.”

“Hippo is a plugin to create stunning vector illustrations and web assets right in Figma, thanks to AI. It’s made to be easy to use, with 22 handpicked styles tailored for the web – no prompt engineering required. It can also vectorize or edit any image with AI.”