How We Built a Raster Painting App in Flutter

Building a professional raster painting application with Flutter presented unique technical challenges that pushed us to innovate beyond traditional approaches.

Building a Raster Painting App with Flutter 🔗

Our journey to build a raster painting application with Flutter was a story of overcoming technical challenges and thinking outside the box. We’ll share our experience, from our initial attempts with a naive approach and platform-specific solutions to our final, innovative “sandwich” approach.

The Initial Hurdle: Flutter’s Vector Nature 🔗

Flutter’s architecture, with its powerful rendering pipeline, is perfect for building beautiful, fast UIs. At its core, Flutter is vector-based. It takes your UI code, processes it through the Flutter Engine, and creates a Vector Drawing List. This list is a series of commands for drawing shapes, lines, and text, which are then rendered into pixels by Skia or Impeller.

Our challenge was that a painting app needs to manipulate pixels directly—a raster-based operation. We couldn’t just add another vector shape to a list with every brush stroke.

We initially chose Flutter for its cross-platform capabilities and decided to use a powerful C++ brush engine, LibMyPaint, to handle the complex painting logic.

The Naive Approach: Asynchronous Image Loading 🔗

Our very first attempt was a simple, yet ultimately flawed, idea. We had LibMyPaint render a brush stroke to a bitmap in memory. We then converted this bitmap to a format that Flutter could understand, such as a byte array, and passed it to the UI thread. There, we would render it using Flutter’s built-in Image.memory widget.

This approach was incredibly slow.

Every time a brush stroke was made, we had to:

1. Render the stroke to a bitmap in our native code.
2. Pass the entire bitmap data back to the Dart side.
3. The Image.memory widget had to decode this data and then display it.

This process was repeated for every single update, creating a massive bottleneck and an extremely laggy user experience. We quickly realized that this method was not viable for a real-time painting application.

Attempt 1: The GL Texture Bridge 🔗

Our first robust idea was to use a platform channel to communicate with a native C++ plugin. The plugin would use LibMyPaint to do the pixel manipulation on a bitmap. This bitmap would then be rendered by Flutter as a GL (OpenGL/WebGL) texture.

Here’s how it worked:

1. Our Dart code sent drawing commands to a native C++ plugin.
2. The plugin used LibMyPaint to paint on a bitmap.
3. The plugin updated a GL texture with the new pixel data.
4. Flutter’s Texture widget then displayed this texture on the screen.

While this worked, it wasn’t ideal. The performance was not up to our expectations, and we faced a significant problem: we had to write and maintain different texture backends for each platform (Android, iOS, Windows, macOS). As a small team, this was a major roadblock.

Attempt 2: The “Sandwich” Approach 🔗

Frustrated with the overhead and platform-specific work, we went back to the drawing board and came up with a radical new idea, which we nicknamed the “sandwich” approach. Instead of treating LibMyPaint as an external library, we decided to integrate it directly into the Flutter Engine itself.

Here’s how we did it:

1. Custom Flutter Build: We created a custom build of the Flutter engine, a C++ codebase, and embedded LibMyPaint within it.
2. Raster Canvas API: We exposed a new Raster Canvas API directly to our Dart code. This API allowed us to issue low-level, pixel-based commands that were immediately handled by LibMyPaint inside the engine.
3. Unified Rendering: Because LibMyPaint was now part of the engine, its output could be directly processed and integrated into the Vector Drawing List. This meant our brush strokes were rendered by Skia/Impeller just like any other Flutter element, but without the overhead of external plugins or texture management.

This approach was a game-changer. It simplified our codebase, dramatically improved performance, and eliminated the need to write platform-specific code for our core painting logic. We created a version of Flutter that was perfectly tailored for raster painting, allowing us to build a powerful cross-platform application with a single, efficient architecture.

The Proof is in the Performance 🔗

The performance difference between our naive approach and the “sandwich” approach is night and day. We invite you to see for yourself how a well-designed architecture can transform a user experience.

Need a team to build your next big thing? At tinisoft, we offer experienced engineering teams for hire to help bring your vision to life. Whether it’s complex mobile applications, custom Flutter solutions, or innovative software architectures, we’re here to help. Reach out to us at contact@tinisoft.in.