counternotions

Adobe has been known to swing to its own unique interface tune on both Mac OS and Windows. Mac users have long complained about Adobe’s considerable disregard of Mac OS native technologies, from the pointed absence of full-fledged AppleScript in Photoshop for over a decade to the absence of 64-bit version of that app on Mac OS X when it will ship for Windows likely next year.

Adobe’s UI convergence

Adobe’s goal has always been to establish its own UI as a cross-platform, third alternative to both Mac and Windows native UIs. In the process, Adobe has not been shy to invent new UI conventions that clash with native UI patterns. A case in point is the rather funky window title/menu bar design approach in the upcoming Dreamweaver, Thermo and Fireworks apps:

It can be argued that the company has been quite successful at it. By the time Creative Suite 4 ships, Adobe will have created UI patterns and a look-and-feel shared by most of its family of desktop content creation apps that work virtually identically on both platforms.

However, when it comes to stretching the boundaries of application interface nothing comes close to Flash, Adobe’s ubiquitous web media runtime. From bizarre “Skip Intros” to enigmatic UI widgets, everyone has a “Can you believe somebody actually designed that?” moment. Well aware of this, Adobe has been steadily expanding a set of (default) UI widgets and components in its Flex framework that has come to define the look and feel of many Rich Internet Applications around the web. Of course, one doesn’t have to use the default Flash/Flex UI controls as they can be varied by creating custom skins:

UI convergence on the mature desktop market may be a given for Adobe, but the next frontier is to establish Flash Lite (Flash’s mobile version) as the premier runtime and application development platform for mobile devices, now dominated by Symbian, Windows Mobile and Java.

Nokia, for one, will be rolling out Flash Lite 3.0 on its S60 series phones this year:

Why not the iPhone?

While some version of Flash Lite is available on millions of phones, it’s conspicuously absent on the iPhone.

Many reasons have been floated for why Flash isn’t a good match for the iPhone: it’s slow, it hogs CPU cycles, it drains the battery, it crashes too often, it’s not optimized for Mac OS X and so on. As obvious as these reasons may be, even if all those technical issues could be solved tomorrow, there would still remain a huge divide between Adobe and Apple on the iPhone: who controls the UI?

Adobe’s conundrum

Adobe’s interface strategy on the desktop has been to avoid using native UI controls on the deployed OS in favor of establishing and leveraging its own unique cross-platform UI paradigm that its runtime engine can deliver consistently on multiple platforms.

Unfortunately for Adobe though, Apple is in the process of establishing the first mass-market, multi-touch platform with the iPhone and has already migrated it partially to its notebooks. Clearly, Apple (which acquired FingerWorks and its patent portfolio in 2005) has greater ambitions in establishing a broader multi-touch gesture paradigm, likely to cover other devices as well:

At its introduction, what immediately distinguished the iPhone among all other mobile devices was the pervasive and consistent application of a multi-touch UI. It was as big a jump from the WIMP (”window, icon, menu, pointer”) desktop metaphor to a direct manipulation paradigm as we’ve seen in the computer and consumer electronics industries.

Not unexpectedly, every other phone manufacturer has already lined up as the next “iPhone killer” with myriad prototypes and a few shipping products. It’s easy to see in the results that it’s quite problematic to duplicate the fluidity of the iPhone’s UI without infringing Apple’s numerous patents or creating groundbreaking advances. Most pretenders aren’t up to the task: here, for example, is a Flash Lite file that “transforms” Nokia S60 phones into iPhones:

Once you get to “secondary” pages (see video), the touch paradigm gives way to the ugly WIMP interface, which has been the common theme in the vast majority of “iPhone killer” attempts. Some competitors even tried to bypass multi-touch altogether. Samsung, for example, patented a rather impractical gesture-based UI where hand/finger gestures are tracked by a phone’s camera for interaction and navigation:

Apple: Not on my device!

With the iPhone, Apple has been slowly but surely creating a third major OS platform and UI aesthetics commensurate with a multi-touch driven, small scale mobile device. This is a company so obsessed with UI and user experience that it hasn’t yet introduced something as basic as copy and paste on the iPhone 2.0 because, as is rumored, it hasn’t quite nailed a proper UI pattern.

In this highly charged and competitive marketplace to establish the next UI paradigm for mobile devices, Apple is not about to give Adobe or any other company free reins to dilute its brand proposition by introducing cross-platform, common-denominator UIs and interaction patterns to be mingled with Apple’s carefully orchestrated multi-touch approach. So what does that leave Adobe with?

Adobe’s iPhone choices

To put Flash on the iPhone Adobe may:

• strike a deal to license Apple’s entire iPhone UI controls and interaction patterns and ship them with Flash, Flex and AIR development suites as components, much like its current default set “Halo.” Apple hasn’t yet shown any inkling that it’s willing go along with this.
• decide to duplicate the iPhone UI and ignore a legal threat from seriously irked Apple IP lawyers. (For a company that once sued Macromedia for UI infringement that would be supremely ironic.)
• leave the task of creating iPhone compatible Flash components and skins to third party partners and let them deal with the legal ramifications.
• put itself in an unenviable position of having one UI paradigm for the iPhone and another for the rest of the devices it runs on, which would mean that a non-trivial application delivered for the Flash Lite runtime would end up being (at least visually) broken either on the iPhone or on other devices. After all, even the simplest of Flash UI patterns like dragging or double-clicking won’t work on the iPhone as they have entirely different consequences on this multi-touch platform.
• abandon Flash’s long standing strategy of using its own “third-platform” UI paradigm distinct from the native look and feel of the OS it’s running on by using OS-native UI controls, thereby creating a litany of OS-specific headaches for designers and developers. After all, a UI designed for the multi-touch iPhone just wouldn’t be conveniently re-morphed to run smoothly on lesser devices and vice versa.
• encourage developers to design and produce separate versions of their Flash apps for the iPhone and other platforms, but that simply defeats the whole purpose of using Flash to deliver seamlessly compliant multi-platform apps.
• develop its own non-infringing multi-touch gesture library for Flash to either compete with Apple on the iPhone, or just ignore the iPhone and try to establish an anti-iPhone multi-touch platform. (In this regard, it may approach Google to make all or parts of its technology available to Android.)

Some of these theoretical choices are of course impractical as third-party applications cannot be legally loaded on iPhones without going through “The App Store” controlled by Apple.

Apple’s Flash choices

By being so different than other platforms, Apple’s multi-touch, direct-manipulation iPhone UI clearly upsets Adobe’s strategy of a distinct, non-native UI paradigm consistent across multiple platforms and devices. With its recent purchase of chip design house PA Semi, Apple has signaled that it’s aiming to further differentiate its iPhone platform by tying native capabilities to custom-designed system-on-chips, unavailable to third parties and cloners alike. It’s not difficult to imagine that various iPhone UI capabilities will be enhanced by the power of Apple-only hardware components. So it will be increasingly difficult for Adobe or others to battle with Apple either subversively on the iPhone or competitively on other devices.

So what about Flash on the iPhone? Can the iPhone succeed without it? That’s a two-pronged challenge for Apple.

When pundits refer to the absence of “Flash” on the iPhone, they usually mean streaming Flash video, the kind that put YouTube on the map. However, that was the easier problem for Apple to (at least partially) solve. Through an agreement with Google, Apple was able to convince the biggest conduit of Flash video online to make its videos available in the widely deployed industry-standard H.264 codec, which also drives QuickTime videos. Not too long after that Adobe switched to H.264 as its primary video codec in Flash Player 9.

The other missing part of Flash is the ubiquitous .swf runtime that delivers Rich Internet Applications in the browser, and now with Adobe AIR on the desktop as well. As we have discussed previously, unlike Adobes’ Flash, Microsoft’s Silverlight and Sun’s JavaFX, Apple doesn’t have a multimedia/RIA runtime. While all three companies have publicly expressed keen interest to put their runtime engine on the iPhone, Apple has shown no interest thus far. Indeed, despite great pundit pressure, Steve Jobs went out of his way to declare Flash “too slow to be useful” on the iPhone and Flash Lite “not capable of being used with the web.”

Apple’s “RIA runtime” is turning out to be WebKit, as we detailed previously. With the upcoming MobileMe apps Apple’s arguing that an Ajax-based UI in a web browser can be as effective as a desktop app or an RIA delivered via Flash. In other words, Apple is saying: if you want to natively deliver an app for the iPhone use Cocoa Touch and if you’re reaching for cross-platform ubiquity (including on the iPhone with Safari mobile) use Ajax. Not a complicated proposition.

As for Flash, last year an Apple developer document (”Optimizing Web Applications and Content for iPhone”) listed just one item under the section “Unsupported Technologies”:

“You’ll want to avoid using Flash and Java for iPhone content. You’ll also want to avoid encouraging users to download the latest Flash on their iPhone, because neither Flash nor downloads are supported by Safari on iPhone.”

Clearly, Adobe has an uphill battle to either re-engineer the Flash runtime for the iPhone or convince Apple to specifically accommodate it by demonstrating an indisputable need for it.

My quest for extraordinary and ephemeral typefaces continue:

Limbtype

Be sure to look at the other “characters” in Flickr.

Ice cream lettering

That’s the first frame of a short video (scroll to the bottom at the site) that reveals what happens to meltable typefaces.

» Via Quipsologies.

Creando conciencia en las calles

Outstanding blending of digital and hand-drawn typography for a thoughtful second-take effect. (More at the site.)

» Via NFG.

Oversized screw as a table by FinnishDesignShop

For when you need to send a signal to your advertising agency or application development shop; nice furniture for close-quarters negotiations.

» Via Sensory Impact.

Walking on exhibit walls

While designing the space for the 2007 collection of 100% Products, Nendo decided to lock the entire exhibit – furniture and all on to the wall surface. The spatial hack was intended to create attention by allowing the viewer to see the exhibit from a floor above. In addition, any visitor the booth would end being part of the exhibit by appearing to be walking on the wall of the room.

» Via Sensory Impact.

In a week, you’ll be able to purchase Leopard. In a few years, you may look at it as a watershed event for a new interface paradigm. With over 300+ features Leopard is a big cat. Here, we’ll only explore key components to the extent they’ll likely influence hardware/software interface design in the next 2-5 years.

Core Animation

Often described as the biggest change to popular PC interfaces since the original Mac thirty years ago, Core Animation (CA) is Apple’s future:

Welcome to the next level in computer animation. No, it’s not a feature film — it’s your desktop. Core Animation is a framework that makes it simple for Mac developers to add visually stunning user interfaces, graphics, and animations to applications.

Scott Forstall, VP of platform experience at Apple, demonstrated the power of CA at the 2006 WWDC (video at 00:49:28 - 00:53-08):

At one extreme, we have pundits who aren’t likely to grok Core Animation like John Dvorak whose first reaction to CA at the June 2007 WWDC was:

Not sure what the Vista comparison is here. The demo reminded me of Microsoft Max photo-sharing application. The WWDC developers attending the Jobs keynote didn’t seem wowed with this functionality.

Then there are developers of Mac apps like TextMate, FlySketch, VoodooPad and Delicious Library who are so giddy about CA that they already decided the next version of their apps will only work with CA on Leopard. Here’s Delicious Library developer Wil Shipley on CA:

…every time you give developers a chance to do better graphics with less code, you’re going to see another revolution in user experience. The revolution coming with Core Animation is akin to the one that came from the original Mac in 1984 — the Mac said “here’s a relatively easy way to add graphics to your user interface” and Core Animation says, “Here’s a very easy way to add composited layers and motion to your interface.”

and on upgrading to DL2:

Yes, Delicious Library 2 is based entirely around Core Animation and other key Leopard technologies, so our customers are going to have to upgrade their OS if they want to upgrade our program… We realized any app we released based on Tiger was going to look really pathetic when Leopard came out.

Direct manipulation and Windowless UI

The potential of CA is a move away from proxy interaction via UI controls to direct manipulation of objects on screen with fluid, ‘natural’ and fun gestures, like flick-scroll and pinch-zoom on the iPhone. Shipley:

What we’ll see with Core Animation is a move away from widgets and into direct manipulation. In Delicious Library 2, we’re conveying much more information directly on our bookshelf view, instead of using textfields and the like, and similarly we’re allowing the user to interact more directly with the books on the shelf, instead of just looking at them and then pressing buttons on another part of a screen to change them.

For some designers, these gestural and animated UI paradigms may look like rather frivolous, GPU-wasting eye-candy. But, just as the iPhone forever changed phone UIs, so will CA on desktops and various mobile and media devices.

What makes CA critical to Apple’s future is the fact that Core Animation is not only in Mac OS X 10.5 but because Leopard is also the foundation of iPhone, iPod touch, AppleTV and likely other products to come, the direct manipulation UI paradigm now moves into a large spectrum of devices.

From tiled windows to windows-inside-of-windows MDI to overlapping Mac windows, we have come full circle. On these post-PC devices there are often no windows — no mouse, pull-down menus or scroll bars either.

Once users are exposed to useful UI paradigms, like instant in-line video playback via Flash or partial page refreshes via Ajax, there’s hardly any going back. As subtle CA effects seep into users’ daily interaction with their screens, we’ll all wonder how we ever lived without them. Here, for instance, is a list of many CA-type UI examples that are already a part of OS X users’ interaction lexicon: drawer, sheets, window minimization, Dock magnification, list shuffling, etc.

While manipulating screen objects via gestures and non-WIMP approaches are not entirely new, this moves from expensive, cumbersome, demo-only, limited-purpose or often Flash-based prototypes to a mainstream OS and daily exposure to tens of millions. Just how powerful an impact this could have can be seen in the explosion of ‘touch-screen’ phones from all sorts of manufacturers following the introduction of the iPhone. Now it’s not so easy to sell a high-end phone without some form touch-screen interface.

With Leopard and the upcoming iPhone SDK for third-party developers, Pandora’s box is wide open. Of course, some will see this as regression into UI chaos. Starting prominently in 1999 with QuickTime 4.0, Apple has thrown open the gates of two decades of Human Interface Guidelines: the ‘lickable’ Aqua, Panther’s brush metal, iLife and pro apps, widgets, the ‘unified-look’ in Tiger, etc. Clearly, Core Animation is a new frontier that all designers will have to adjust to.

Next, we’ll take a strategic look at other Leopard technologies that will affect designers, including virtualization, Time Machine, resolution independent UIs, widgets and others. Please come back or subscribe to the RSS feed.