“Ordered Information” is not a paint job

At Davos 2013, CEO Marissa Mayer unveiled her vision for Yahoo’s rebirth, affirming it wants to be:

a feed of information that is ordered, the Web is ordered for you and is also on your mobile phone.

It’s a laudable vision. With the decline of traditional gatekeepers of information, the original directory of the Information Superhighway could conceivably become again an “orderer” of the abundance of information we live in. Yahoo also has an abundance of mobiles apps, on iOS alone: Search, Flickr, Mail, Finance, News, IntoNow, Messenger, TimeTraveler, Axis, MarketDash, Sportacular, Fantasy Baseball, Basketball, Hockey, Cricket and Football, as well as feeds into Siri queries and built-in Weather and Stocks apps.

It would thus appear Yahoo both “has content” and “gets mobile.” Unfortunately, when dealing with information for the past 15 years, Yahoo has been confusing “ordered information” with “content aggregation” and “task completion.” While ordered information is presumably better than unordered information, simple aggregation — and that’s all Yahoo has, for the most part — is still quite low on the online food chain, both for Yahoo’s bottom line as well as utility to its users.

What then for a company like Yahoo — cyclically self-described as a search, media, technology, portal and advertising company — could go wrong in a rebirth process? Let’s take a page from Yahoo! Finance, a property that has long dominated its space against old (Microsoft, AOL) and relatively new (Morningstar, Google) competitors, for it neatly displays the degree of trouble Yahoo is in.

Yahoo! Finance

Is it “ordered”?

Scanning top-to-bottom a page of stock quote for AAPL, there are three disparately located Search departure points [1, 2, 3] all styled differently: [1] “Search Web” is the most emphasized one of the three. And yet it’s for generic search, even though I’m already at a highly specialized finance site.


[2] “Get Quotes” is finance-specific, but then [3], not a button but a text-link, is named “Finance Search” and yet, inexplicably, takes you to another page with a completely different UI and a generic searchbox with text-links to images, video, shopping, etc., not unlike the giant generic “Search Web” [1] just above it. Baffling for a company that still harbors search aspirations.


[4] Instead of a single banner ad, there are four small ad-buttons for competing trading tools right next to each other. I’m sure some people click on those, but what a waste of prime real estate for premium monetization.

[5] If you wanted to see the price of the stock you’re currently looking at in the context of Dow and Nasdaq as people would want to do, well, you’re out of luck. It’s to the left, on top the navigation column, above the ads.

[6] Why on earth is it necessary for Yahoo to display a Facebook Like button with running tally on an individual stock quote page? Believing Facebook Connect or Like can somehow be Yahoo’s friend in the long run is self-destructively naive.

[7] It’s a navigation column with a kitchen-sink attitude, one of half a dozen other section tabs on top of the page, likely bewildering for most casual users of the site and not quite comprehensive enough for professional traders.


[8] You’d think “customize chart” (unlike other text-links, all lower-case) would allow you to customize the small chart in some meaningful way. Click it and you’re taken to, you guessed it, another page with a completely different, barren UI from the 1950s. It allows you to change nothing but the timespan of the chart to either 1 day, 5 days or 1 year. That’s customization for you, one which was available on the original chart itself to begin with, without having to navigate to another page under false pretenses.

[9] There comes the positively Amazon-ian “People viewing {current stock} also viewed:” inanity. People who love/hate AAPL apparently also love/hate/crave Chipotle and MasterCard. Seriously?

[10] I don’t have access to click rates on that “Trade Now” button, obviously, but I’m guessing those who do click are a small minority. Clearly this linkage to brokers is meant to generate significant origination revenue for Yahoo, but all those broker sites also provide roughly equal or better generic financial info. Somebody who trades at Schwab or Fidelity wouldn’t necessarily be wasting time at Yahoo, while trading.

[11] Also, don’t plan on having more than one broker to quickly go to, since you can only have one.

[12] I’m not entirely sure whether people actually compare brokers at Yahoo! Finance, but even if they do, they get to choose from only four pre-selected brokers, presumably paying Yahoo an origination fee.


[13] Just two articles “featured”? What’s the logic behind featuring random articles here, other than cheap SEO linkage?

[14] One of the two articles “featured” is about RIM’s upcoming BlackBerry 10 phone. I know Eric Jackson has plenty of interesting things to say about AAPL, and does pretty much every single day, but this featured piece has absolutely nothing to do with Apple, the word doesn’t even appear anywhere in the linked article or the accompanying video. It is about RIM, as the title says, a company Apple hasn’t really been competing against for nearly two years now.

[15] (I’m not logged into Finance here but) I’m a proud resident of NYC without a car and yet this is a localized car service ad, from another state. Normally, I’d never see any of these ads, as I use an ad blocker.

[16] A river of news. Really, any text that seems to contain the words “Apple” or “AAPL” can end up here and does. No hierarchy or grouping. The “Filter Headlines” text-link on top takes you to another page with, yes, a completely different, barren UI from the 1950s that allows you to check/uncheck a uneditable list of third party news sources.

[17] Sector/Industry/Sub-Industry categorization of US stocks is an issue for nearly all finance services but, newsflash, Apple dropped “Computer” from its name over 5 years ago. Beyond semantics, this is completely nontrivial for a finance site. Readers shouldn’t be led to believe that nothing more than a small and declining percentage of Apple’s revenue comes from selling “Personal Computers.”

[18] As can be seen in the second column here, Apple’s market cap is 20 times Dell’s. They’re in different leagues and (except for desktops and notebooks) in totally different markets. Apple has stopped worrying about Dell many years ago. Since the title of the section reads “Comparison” (singular), of all Apple’s fierce competitors, Yahoo picked Dell? Furthermore, if you do click the text-link “More Competitors” (plural) you’re taken to another baffling page with random stuff thrown in, but with only a single competitor shown: you guessed it, Dell.


[19] If the stock you’re investigating is an unfamiliar one, wouldn’t it be useful to see what the company does right on top? The “Business Summary” that describes what Apple does is way down. If you were in Wall Street looking for AAPL, it’d be across the river in Jersey City.


[20] “Penny Stocks.” That cheesy ad pretty much says it all.

[21] Live by 3-column layouts, die by trailing, unbalanced empty space.

[22] “Sponsored Links”? Be respectful of the user, just call them “Ads,” especially if you are an advertising company. If you’re ashamed of carrying advertising detritus at the very bottom of your site, 50 feet below sea level, maybe it’s time to rethink the longer prospects of revenue generation at Yahoo! Finance beyond penny stocks and car mechanics?


[23] The long list of data feeders to Yahoo is telling. Yahoo! Finance is obviously an aggregator and not a smart or a handsome one at that. Yahoo doesn’t generate all this stuff, you might say, companies that provide the data feeds to Yahoo do. For a company that wants to “order the information,” that is the problem, isn’t it?

A problem deeper than UI

In the latest quarter earnings call Marissa Mayer gave further indication of how this “ordered information” vision might be deployed:

Overall in search, it’s a key area of investment for us. We need to invest in a lot of interface improvements. All of the innovations in search are going to happen at the user interface level moving forward and we need to invest in those features both on the desktop and on mobile and I think both ultimately will be key plays for us.

If Yahoo wants simply to be the data provider to gatekeepers like Apple, Facebook and Google, it will be problematic since Yahoo is often not the originator of the data flow [23]. General-purpose feed business is not a high-margin opportunity in any case. As the simple analysis above shows when the underlying data layer is decoupled from the UX, especially at the hands of designers insufficiently enthused, the results are mediocre at best. In the long run, it’s difficult to “pretty up” data that you don’t control and derive meaningful profit/benefit from, ask Apple’s Maps team.

And yet, this is Finance, one of Yahoo’s strongest properties, not a minor beta product. It’s enough of an embarrassing UX debacle that I haven’t even mentioned any of the glaring visual design issues at all. No amount of interface pixel-dust will cover up the fundamentally broken opportunity to make financial information actually useful to consumers and semi-professionals, while making money doing it. For the user, it fails the first test: now that I’m presented with all this aggregated information, what do I do with it?

Product design starts right there: What Do I Do With It? Not at picking one of 41 shades of purple. Ordered information is great but hard…repainted information is cheap but insufficient. Here’s hoping Yahoo gets that this time around.


P.S. I don’t mean to single out Yahoo here. All the other financial sites, like Bloomberg or Reuters, also suffer from similar misunderstandings of how to make financial information usable and useful. I picked Yahoo because Marissa Mayer appears to be actively revamping various properties, Yahoo! Finance desperately needs TLC and I’d like to see Yahoo come out of its funk if for no other reason than as a balance to Google’s growing dominance in online services. This is clearly a things-I-noticed in-20-minutes kind of a superficial treatment. I simply scanned one stock quote page of one of the myriad sections at Yahoo! Finance. The list of things-that-went-wrong around the entire site must be enormous. It’s also obvious that Yahoo! Finance wasn’t made with passion, focus and much attention to detail, like a lot of Yahoo properties. That said, providing product design analysis from outside a company, without knowledge of its business goals, is fraught with danger So I’ll have to refrain from offering gratuitous advice to Yahoo on how to fundamentally reformulate their web and mobile Finance presence.

How many iDownloaders does it take to screw an App Store?

Search results for “iDownloader” at Apple App Store:


Yes, it’s a big operation. Yes, there’ll be a million apps soon. Yes, many apps will inevitably be similar. Yes, shady developers steal other developers’ IP. Yes, all sorts of people try to game it. Yes, the power law may apply to revenues. Yes, you can’t please everyone all the time. Yes, other app stores may be worse. Yes, the App Store was once the crown jewel of Apple’s mobile empire.

Yes, there are many ways to spin this… None fits the bill as much as the notion that Apple’s inability or unwillingness to fundamentally improve categorization, discovery, navigation, display, promotion, fraud, pricing and reviews at the App Store has been most glaring.

Chomp change, indeed.

Can Siri go deaf, mute and blind?

Earlier in “Is Siri really Apple’s future?” I outlined Siri’s strategic promise as a transition from procedural search to task completion and transactions. This time, I’ll explore that future in the context of two emerging trends:

  • Internet of Things is about objects as simple as RFID chips slapped on shipping containers and as vital as artificial organs sending and receiving signals to operate properly inside our bodies. It’s about the connectivity of computing objects without direct human intervention.
  • The best interface is no interface is about objects and tools that we interact with that no longer require elaborate or even minimal user interfaces to get things done. Like self-opening doors, it’s about giving form to objects so that their user interface is hidden in their user experience.

Apple’s strength has always been the hardware and software it creates that we love to carry, touch, interact with and talk about lovingly — above their mere utility — like jewelry, as Jony Ive calls it. So, at first, it seems these two trends — objects talking to each other and objects without discernible UIs — constitute a potential danger for Apple, which thrives on design of human touch and attention. What happens to Apple’s design advantage in an age of objects performing simple discreet tasks or “intuiting” and brokering our next command among themselves without the need for our touch or gaze? Indeed, what happens to UI design, in general, in an ocean of “interface-less” objects inter-networked ubiquitously?

Looks good, sounds better

Fortunately, though a star in her own right, Siri isn’t wedded to the screen. Even though she speaks in many tongues, Siri doesn’t need to speak (or listen, for that matter) to go about her business, either. Yes, Siri uses interface props like fancy cards, torn printouts, maps and a personable voice, but what makes Siri different is neither visuals nor voice.

Despite the knee-jerk reaction to Siri as “voice recognition for search,” Siri isn’t really about voice. In fact, I’d venture to guess Siri initially didn’t even have a voice. Siri’s more significant promise is about correlation, decisioning, task completion and transaction. The fact that Siri has a sassy “voice” (unlike her competitors) is just endearing “attitude”.


Those who are enthusiastic about Siri see her eventually infiltrating many gadgets around us. Often seen liaising with celebrities on TV, Siri is thought to be a shoo-in for the Apple TV interface Oscars, maybe even licensed to other TV manufacturers, for example. And yet the question remains, is Siri too high maintenance? When the most expensive BOM item in an iPhone 5 is the touchscreen at $44, nearly 1/4 costlier than the next item, can Siri afford to live outside of an iPhone without her audio-visual appeal?

Well, she already has. Siri Eyes Free integration is coming to nine automakers early this year, allowing drivers to interact with Siri without having to use the connected iPhone screen.


Given Siri Eyes Free, it’s not that difficult to imagine Siri Touch Free (see and talk but not touch), Siri Talk Free (see and touch but not talk) and so on. People who are impatient with Apple’s often lethargic roll out plans have already imagined Siri in all sorts of places, from aircraft cockpits to smart wristwatches to its rightful place next to an Apple TV.

Over the last decade, enterprise has spent billions to get their “business intelligence” infrastructure to answer analysts’ questions against massive databases from months to weeks to days to hours and even minutes. Now imagine an analyst querying that data by having a “natural” conversation with Siri, orchestrating some future Hadoop setup, continuously relaying nested, iterative questions funneled towards an answer, in real time. Imagine a doctor or a lawyer querying case histories by “conversing” with Siri. Forget voice, imagine Siri’s semantic layer responding to 3D gestures or touches on glass or any sensitized surface. Set aside active participation of a “user” and imagine a monitor with Siri reading microexpressions of a sleeping or crying baby and automatically vocalizing appropriate responses or simply rocking the cradle faster.

Scenarios abound, but can Siri really afford to go fully “embedded”?

There is some precedence. Apple has already created relatively successful devices by eliminating major UI affordances, perhaps best exemplified by the iPod nano ($149) that can become an iPod shuffle ($49) by losing its multitouch screen, made possible by the software magic of Genius, multi-lingual VoiceOver, shuffle, etc. In fact, the iPod shuffle wouldn’t need any buttons whatsoever, save for on/off, if Siri were embedded in it. Any audio functionality it currently has, and much more, could be controlled bi-directionally with ease, in all instances where Siri were functional and socially acceptable. 3G radio plus embedded Siri could also turn that tiny gadget into so many people’s dream of a sub-$100 iPhone.


Grounding Siri

Unfortunately, embedding Siri in devices that look like they may be great targets for Siri functionality isn’t without issues:

  • Offline — Although Siri requires a certain minimum horsepower to do its magic, much of that is spent ingesting and prepping audio to be transmitted to Apple’s servers which do the heavy lifting. Bringing that processing down to an embedded device that doesn’t require a constant connection to Apple maybe computationally feasible. However, Apple’s ability to advance Siri’s voice input decoding accuracy and pattern recognition depend on constant sampling of and adjusting input from tens of millions of Siri users. This would rule out Siri embedded into offline devices and create significant storage and syncing problems with seldom-connected devices.
  • Sensors — One of the key reasons why Siri is such a good fit for smartphones is the number of on-device sensors and the virtually unlimited range of apps it’s surrounded with. Siri is capable of “knowing” not only that you’re walking, but that you’ve also been walking wobbly, for 35 minutes, late at night, in a dark alley, around a dangerous part of a city, alone… and send a pre-designated alert silently on your behalf. While we haven’t seen examples of such deep integration from Apple yet, Siri embedded into devices that lack multiple sensors and apps would severely limit its potential utility.
  • Data — Siri’s utility is directly indexed to her access to data sources and, at this stage, third parties’ search (Yelp), computation (WolframAlpha) and transaction (OpenTable) facilities. Apple does and is expected to continue to add such partners in different domains on a regular basis. Siri embedded in radio-lacking devices that don’t have access to such data and processing, therefore, may be too crippled to be of interest.
  • Fragmentation — People expect to see Siri pop up in all sorts of places and Apple has taken the first step with Siri Eyes Free where Siri gives up her screen to capture the automotive industry. If Siri can drive in a car, does that also mean she can fly on an airplane, sail on a boat or ride on a train? Can she control a TV? Fit inside a wristwatch? Or a refrigerator? While Siri — being software — can technically inhabit anything with a CPU in it, the radio in a device is far more important to Siri than its CPU, for without connecting to Apple (and third party) servers, her utility is severely diminished.
  • Branding — Siri Eyes Free won’t light up the iPhone screen or respond to commands that would require displaying a webpage as an answer. What look like reasonable restrictions on Siri’s capabilities in this context shouldn’t, however, necessarily signal that Apple would create “subsets” of Siri for different domains. More people will use and become accustomed to Siri’s capabilities in iPhones than any other context. Degrading that familiarity significantly just to capture smaller markets wouldn’t be in Apple’s playbook. Instead of trying to embed Siri in everything in sight and thus diluting its brand equity, Apple would likely pair Siri with potential NFC or Bluetooth interfaces to devices in proximity.

What’s Act II for Siri?

In Siri’s debut, Apple has harvested the lowest hanging fruit and teamed up with just a handful of already available data services like Yelp and WolframAlpha, but has not really taken full advantage of on-device data, sensor input or other novel information.

As seen from outside, Siri’s progress at Apple has been slow, especially compared to Google that has had to play catch up. But Google must recognize a strategically indispensable weapon in Google Now (a Siri-for-Android, for all practical purposes) as a hook to those Android device manufacturers that would prefer to bypass Google’s ecosystem. None of them can do anything like it for some time to come, Samsung’s subpar attempts aside.

If you thought Maps was hard, injecting relationship metadata into Siri — fact by fact, domain by domain — is likely an order of magnitude more laborious, so Apple’s got her work cut out for Siri. It’d be prudent not to expect Apple to rush into embedding Siri in its non-signature devices just yet.

Is Siri really Apple’s future?

Siri is a promise. A promise of a new computing environment, enormously empowering to the ordinary user, a new paradigm in our evolving relationship with machines. Siri could change Apple’s fortunes like iTunes and App Store…or end up being like the useful-but-inessential FaceTime or the essential-but-difficult Maps or the desirable-but-dead Ping. After spending hundreds of millions on acquiring and improving it, what does Apple expect to gain from Siri, at once the butt of late-night TV jokes but also the wonder of teary-eyed TV commercials?

Everyone expects different things from Siri. Some think top 5 wishes for Siri should include the ability to change iPhone settings. The impatient already think Siri should have become the omniscient Knowledge Navigator by now. And of course, the favorite pastime of Siri commentators is comparing her query output to Google Search results while giggling.

Siri isn’t a sexy librarian

The Google comparison, while expected and fun, is misplaced. It’d be very hard for Siri (or Bing or Facebook, for that matter) to beat Google at conventional Command Line Interface search given its intense and admirable algorithmic tuning and enormous infrastructure buildup for a decade. Fortunately for competitors, though, Google Search has an Achilles heel: you have to tell Google your intent and essentially instruct the CLI to construct and carry out the search. If you wanted to find a vegetarian restaurant in Quincy, Massachusetts within a price range of $25-$85 and you were a Google Search ninja, you could manually enter a very specific keyword sequence: “restaurant vegetarian quincy ma $25…$85″ and still get “about 147,000 results (0.44 seconds)” to parse from. [All examples hereon are grossly simplified.]


This is a directed navigation system around The Universal Set — the entirety of the Internet. The user has to essentially tell Google his intent one. word. at. a. time and the search engine progressively filters the universal set with each keyword from billions of “pages” to a much smaller set of documents that are left for the user to select the final answer from.

Passive intelligence

Our computing devices, however, are far more “self-aware” circa 2012. A mobile device, for instance, is considerably more capable of passive intelligence thanks to its GPS, cameras, microphone, radios, gyroscope, myriad other in-device sensors, and dozens of dedicated apps, from finance to games, that know about the user enough to dramatically reduce the number of unknowns…if only all these input and sensing data could somehow be integrated.

Siri’s opportunity here to win the hearts and minds of users is to change the rules of the game from relatively rigid, linear and largely decontextualized CLI search towards a much more humane approach where the user declares his intent but doesn’t have to tell Siri how do it every step of the way. The user starts a spoken conversation with Siri, and Siri puts an impressive array of services together in the background:

  • precise location, time and task awareness derived from the (mobile) device,
  • speech-to-text, text-to-speech, text-to-intent and dialog flow processing,
  • semantic data, services APIs, task and domain models, and
  • personal and social network data integration.

Let’s look at the contrast more closely. Suppose you tell Siri:

“Remind me when I get to the office to make reservations at a restaurant for mom’s birthday and email me the best way to get to her house.”

Siri already knows enough to integrate Contacts, Calendar, GPS, geo-fencing, Maps, traffic, Mail, Yelp and Open Table apps and services to complete the overall task. A CLI search engine like Google’s could complete only some these and only with a lot of keyword and coordination help from the user. Now lets change “a restaurant” above to “a nice Asian restaurant”:

“Remind me when I get to the office to make reservations at a nice Asian restaurant for mom’s birthday and email me the best way to get to her house.”

“Asian” is easy, as any restaurant-related service would make at least a rough attempt to classify eateries by cuisine. But what about “nice”? What does “nice” mean in this context?

A conventional search engine like Google’s would execute a fairly straight forward search for the presence of “nice” in the text of restaurant reviews available to it (that’s why Google bought Zagat), and perhaps go the extra step of doing a “nice AND (romantic OR birthday OR celebration)” compound search to throw in potentially related words. Since search terms can’t be hand-tuned for an infinite number of domains, this comes into play for highly searched categories like finance, travel, electronics, automobiles, etc. In other words, if you’re searching for airline tickets or hotel rooms, the universe of relevant terms is finite, small and well understood. Goat shearing or olive-seed spitting contests, on the other hand, may not benefit as much from such careful human taxonomic curation.

Context is everything

And yet even when a conventional search engine can correlate “nice” with “romantic” or “cozy” to better filter Asian restaurants, it won’t matter to you if you cannot afford it. Google doesn’t have access to your current bank account, budget or spending habits. So for the restaurant recommendation to be truly useful, it would make sense for it to start at least in a range you could afford, say $$-$$$, but not $$$$ and up.

Therein comes the web browser vs. apps unholy war. A conventional search engine like Google has to maintain an unpalatable level of click-stream snooping to track your financial transactions to build your purchasing profile. That’s not easy (likely illegal on several continents) especially if you’re not constantly using Google Play or Google Wallet, for example. While your credit card history or your bank account is opaque to Google, your Amex or Chase app has all that info. If you allow Siri to securely link to such apps on your iPhone, because this is a highly selective request and you trust Siri/Apple, your app and/or Siri can actually interpret what “nice” is within your budget: up to $85 this month and certainly not in the $150-$250 range and not a $25 hole-in-the wall Chinese restaurant either because it’s your mother’s birthday.

Speaking of your mother, her entry in your Contacts app has a custom field next to “Birthday” called “Food” which lists: “Asian,” “Steak,” and “Rishi Organic White Tea”. On the other hand, Google has no idea, but your Yelp app has 37 restaurants bookmarked by you and every single one is vegetarian. Your mother may not care, but you need a vegetarian restaurant. Siri can do a proper mapping of the two sets of “likes” and find a mutually agreeable choice at their intersection.

So a simple search went from “a restaurant” to “a nice Asian vegetarian restaurant I can afford” because Siri already knew (as in, she can find out on demand) about your cuisine preference and your mother’s and your ability to pay:

Restaurant chain

Mind you, all these series of data lookups and rule arbitrations among multiple apps happen in milliseconds. Quite a bit of your personal info is cached at Apple servers and the vast majority of data lookups in third party apps are highly structured and available in a format Siri has learned (by commercial agreement between companies) to directly consume. Still, the degree of coordination underneath Siri’s reassuring voice is utterly nontrivial. And given the clever “personality” Siri comes with, it sounds like pure magic to ordinary users.

The transactional chain

In theory, Siri’s execution chains can be arbitrarily long. Let’s consider a generic Siri request:

Check weather at and daily traffic conditions to an event at a specific location, only if my calendar and my wife’s shared calendar are open and tickets are available for under $50 for tomorrow evening.

Siri would parse it semantically as:


and translate into an execution chain by apps and services:


Further, being an integral part of iOS and having programmatic access to third party applications on demand, Siri is fully capable of executing a fictional request like:

Transfer money to purchase two tickets, move receipt to Passbook, alert in own calendar, email wife, and update shared calendar, then text baby sitter to book her, and remind me later.

by translating it into a transactional chain, with bundled and 3rd party apps and services acting upon verbs and nouns:


By parsing a “natural language” request lexically into structural subject-predicate-object parts semantically, Siri can not only find documents and facts (like Google) but also execute stated or implied actions with granted authority. The ability to form deep semantic lookups, integrate information from multiple sources, devices and 3rd party apps, perform rules arbitration and execute transactions on behalf of the user elevates Siri from a schoolmarmish librarian (à la Google Search) into an indispensable butler, with privileges.

The future is Siri and Google knows it

After indexing 40 billion pages and their PageRank, legacy search has largely run its course. That’s why you see Google, for example, buying the world’s largest airline search company ITA, restaurant rating service Zagat, and cloning Yelp/Foursquare with Google Places, Amazon with Google Shopping, iTunes and App Store with Google Play, Groupon with Google Offers, Hotels.com with Google Hotel Finder…and, ultimately, Siri with Google Now. Google has to accumulate domain specific data, knowledge and expertise to better disambiguate users’ intent in search. Terms, phrases, names, lemmas, derivations, synonyms, conventions, places, concepts, user reviews and comments…all within a given domain help enormously to resolve issues of context, scope and intent.

Whether surfaced in Search results or Now, Google is indeed furiously building a semantic engine underneath many of its key services. “Normal search results” at Google are now almost an afterthought once you go past the various Google and third party (overt and covert) promoted services. Google has been giving Siri-like answers directly instead of providing interminable links. If you searched for “Yankees” in the middle of the MLB playoffs, you got real-time scores by inning, first and foremost, not the history of the club, the new stadium, etc.

Siri, a high-maintenance lady?

Google has spent enormous amounts of money on an army of PhDs, algorithm design, servers, data centers and constant refinements to create a global search platform. The ROI on search in terms of advertising revenue has been unparalleled in internet history. Apple’s investment in Siri has a much shorter history and far smaller visible footprint. While it’d be suicidal for Apple to attack Google Search in the realm of finding things, can Apple sustainably grow Siri to its fruition nevertheless? Very few projects at Apple that don’t manage to at least provide for their own upkeep tend to survive. Given Apple’s tenuous relationship with direct advertising, is there another business model for Siri?

By 2014, Apple will likely have about 500 million users with access to Siri. If Apple could get half of that user base to generate just a dozen Siri-originated transactions per month (say, worth on average $1 each, with a 30% cut), that would be roughly a $1 billion business. Optimistically, the average transaction could be much more than $1 or the number of Siri transactions much higher than 12/month/user or Siri usage more than 50% of iOS users, especially if Siri were to open to 3rd party apps. While these assumptions are obviously imaginary, even under the most conservative conditions, transactional revenue could be considerable. Let’s recall that, even within its media-only coverage, iTunes has now become a $8 billion business.

As Siri moves up the value chain from its original CLI-centric simplicity prior to Apple acquisition to its current status of speech recognition-dictation-search to a more conversationalist interface focused on transactional task completion, she becomes far more interesting and accessible to hundreds of millions of non-computer savvy users.

Siri as a transaction machine

A transactional Siri has the seeds to shake up the $500 billion global advertising industry. For a consumer with intent to purchase, the ideal input comes close to “pure” information, as opposed to ephemeral ad impression or a series of search results which need to be parsed by the user. Siri, well-oiled by the very rich contextual awareness of a personal mobile device, could deliver “pure” information with unmatched relevance at the time it’s most needed. Eliminating all intermediaries, Siri could “deliver” a customer directly to a vendor, ready for a transaction Apple doesn’t have to get involved in. Siri simply matches intent and offer more accurately, voluntarily and accountably than any other method at scale that we’ve ever seen.

Another advantage of Siri transactions over display and textual advertising is the fact that what’s transacted doesn’t have to be money. It could be discounts, Passbook coupons, frequent mileage, virtual goods, leader-board rankings, check-in credits, credit card points, iTunes gifts, school course credits and so on. Further, Siri doesn’t even need an interactive screen to communicate and complete tasks. With Eyes Free, Apple’s bringing Siri to voice controlled systems, first in cars, then perhaps to other embedded environments that don’t need a visual UI. Apple having the largest and the most lucrative app and content ecosystem on the planet with half a billion users with as many credit card accounts would make the nature of Siri “transactions” an entirely different value proposition to both users and commercial entities.

Siri, too early, too late or merely in progress?

And yet with all that promise, Siri’s future is not a certain one. A few potential barriers stand out:

  • Performance — Siri works mostly in the cloud, so any latency or network disruption renders it useless. It’s hard to overcome this limitation since domain knowledge must be aggregated from millions of users and coordinated with partners’ servers in the cloud.
  • Context — Siri’s promise is not only lexical, but also contextual across countless domains. Eventually, Siri has to understand many languages in over 100 countries where Apple sells iOS devices and navigate the extremely tricky maze of cultural differences and local data/service providers.
  • Partners — Choosing data providers, especially overseas, and maintaining quality control is nontrivial. Apple should also expect bidding wars for partner data, from Google and other competitors.
  • Scope — As Siri becomes more prominent, so grow expectations over its accuracy. Apple is carefully and slowly adding popular domains to Siri coverage, but the “Why can’t Siri answer my question in my {esoteric field}?” refrain is sure to erupt.
  • Operations — As Siri operations grow, Apple will have to seriously increase its staffing levels, not only for engineers from the very small semantic search and AI worlds, but also in the data acquisition, entry and correction processes, as well as business development and sales departments.
  • Leadership — Post-acquisition, two co-founders of Siri have left Apple, although another one, Tom Gruber, remains. Apple recently hired William Stasior, CEO of Amazon A9 search engine, to lead Siri. However, Siri needs as much engineering attention as data partnership building, but Stasior’s A9 is an older search engine different from Siri’s semantic platform.
  • API — Clearly, third party developers want and expect Apple someday to provide an API to Siri. Third party access to Siri is both a gold mine and a minefield, for Apple. Since same/similar data can be supplied via many third parties, access arbitrage could easily become an operational, technical and even legal quagmire.
  • Regulation — A notably successful Siri would mean a bevy of competitors likely to petition DoJ, FTC, FCC here and counterparts in Europe to intervene and slow down Apple with bundling/access accusations until they can catch up.

Obviously, no new platform as far-reaching as Siri comes without issues and risks. It also doesn’t help that the two commercial online successes Apple has had, iTunes and App Store, were done in another era of technology and still contain vestiges of many operational shortcomings. More recent efforts such as MobileMe, Ping, Game Center, iCloud, iTunes Match, Passbook, etc., have been less than stellar. Regardless, Siri stands as a monumental opportunity both for Apple as a transactional money machine and for its users as a new paradigm of discovery and task completion more approachable than any we’ve seen to date. In the end, Siri is Apple’s game to lose.

Apple’s design problems aren’t skeuomorphic

From Tim Cook’s letter announcing the latest reorganization at Apple last week:

Jony Ive will provide leadership and direction for Human Interface (HI) across the company in addition to his longtime role as the leader of Industrial Design. Jony has an incredible design aesthetic and has been the driving force behind the look and feel of our products for more than a decade. The face of many of our products is our software and the extension of Jony’s skills into this area will widen the gap between Apple and our competition.

Sir Jony

Sir Jony Ive needs no introduction

Ive’s industrial design work has been one of the key drivers of Apple’s rebirth. His relentless, iterative focus on simplification of form and function under an aesthetic budget is legend. From Bondi blue iMac to iconic iPods to flatscreen iMacs to iPhones to iPads, his signature is unmistakable.

What’s not publicly known is Ive’s role, if any, on Apple software. The current meme of Ive coming on a white horse to rescue geeks in distress from Scott Forstallian skeuomorphism is wishfully hilarious. Like industrial design of physical devices, software is part form and part function: aesthetics and experience. Apple’s software problems aren’t dark linen, Corinthian leather or torn paper. In fact, Apple’s software problems aren’t much about aesthetics at all…they are mostly about experience. To paraphrase Ive’s former boss, Apple’s software problems aren’t about how they look, but how they work. Sometimes — sadly more often than we expect — they don’t:

  • Notifications, dark linen background or not, is woefully under-designed.
  • Six items that drain mobile device batteries (GPS, WiFi, cellular radio, Bluetooth, notifications and screen brightness) still require laborious, multiple clicks in multiple places, not immediately obvious to non-savvy users to turn on and off, without any simple, thematic or geo-fenced grouping.
  • iCloud-desktop integration and direct file sharing among Apple devices are circuitous and short of “It Just Works.”
  • Many Apple apps, like the iWork suite, are begging to be updated. Others, like Preview, TextEdit, Contacts, desperately need UI and UX overhauls.
  • Core functionalities like the Dictionary or the iOS keyboard layout and auto-correction are not the best of breed.
  • iOS app organization in tiny “folders” containing microscopic icons on pages without names borders on illegible and unscalable.
  • Navigating among running iOS apps (inelegant and opaque for users) and data interchange among apps in general (vastly underpowered for developers) remain a serious problem.

Obviously, it’s not much use piling up on this list, as everyone else’s list of “things to be improved” is likely ten times longer. Neither is it really useful arguing at this point whose fault it is. Apple software — especially its self-declared future, iOS — needs some serious overhaul both in aesthetics and experience, and far more in the latter department.

One Man. One Company. One Aesthetics?

The question is, can one person, even the world’s most eminent industrial designer, pull it off? Is it possible for one person to have enough time in a day to pay sufficient attention to hardware and software — aesthetics and experience — at a level of detail that has become necessary?

After all, Apple’s Human Interaction Guidelines (HIG) has never been just about the aesthetics of icon shadows or button alignment, but also about the behavioral aspects of application design in general. A generation ago, especially prior to the ascendency of web design, HIG was far more respected and adhered to both by Apple itself and its developers. Loyal users also noticed deviations and complained. HIG debates on public forums were not uncommon.

Today, not so much. A radio button or a checkbox can initiate a webpage navigation. Navigational menus now come in circular and triangular popups. There are now purely gestural UIs otherwise betraying none of their functionality to the user. Layers of sliding panels cascade on each other. List items slide left and right to initiate drill-down actions, up and down to reveal interactive media. Some UIs are beveled in 3D, some flat with no shadows, most a loose melange of many styles. And with each such “innovation” they bury the notion of a once powerful HIG a foot deeper.

Is it possible then to have a Human Interface czar for a 500-million user ecosystem today at all? If it were possible, would it be desirable? And if it were possible and desirable, can one person be in charge of both the visual aesthetics and the functional experience of such a huge ecosystem?

  • Can one person truly understand how Siri’s problems surfaced at the display layer actually go deeper into its semantic underpinnings, phoneme parsing, lexical contextuality, data-provider contracts, network latencies, etc., and therefore how the overall solution depends on this interplay of form and function?
  • Is it fair and reasonable to expect one person to understand how user-facing issues that surface within Maps or Passbook apps also suffer from similar technical and operational constraints?
  • Or how a lack of a social layer in Game Center or a content discovery layer in iTunes or App Store impede their functions in so many ways?
  • How about the cognitive mess that is document management and sharing in iClouds, as Apple moves away from user-level file management?
  • Or that monumental experiment also known as the Grand iTunes Redesign that’s been threatening to arrive any year now?
  • Or the Apple TV that needs an injection of a barrel of aesthetics and experience redesign?

In just how many UI corners and UX paths can a HI czar discover the depth of lingering problems and craft solutions as Apple designers play chicken with OS X menubar and iOS status bar translucency and color with every update? These are not just, or even mostly, aesthetic problems.

Apple, quo vadis?

It’s not known if Ive’s is a transitionary appointment necessitated by Scott Forstall’s departure or a harbinger of a longer term realignment of Apple design under a single umbrella. Unification of hardware and software design under a czar may certainly bring aesthetic efficiencies but it can also be pregnant with dangers. Much as the “lickable” Aqua UI ended up doing a decade ago, a serious mistake would be to hide many of these behavioral, functional and experiential software problems under a more attractive, aesthetically unifying display layer, such as:

  • A more modern, less cheesy Game Center redesign that still doesn’t have a social layer.
  • An aesthetically unified iTunes without appreciably better content discoverability.
  • A Siri app without the background linen but still lacking much deeper semantic integration with the rest of the iOS.
  • A Maps app without the ungainly surreal visual artifacts but still missing a robust search layer underneath.
  • An iBooks app without the wooden shelves or inner spine shadow, but still with subpar typography and anemic hyphenation and justification.
  • A Podcast app without the tape deck skeuomorphism, but with all the same navigational opaqueness.

In the end, what’s wrong with iOS isn’t the dark linen behind the app icons at the bottom of the screen, but the fact that iOS ought to have much better inter-application management and navigation than users fiddling with tiny icons. I’m fairly sure most Apple users would gladly continue to use what are supposed to be skeuomorphically challenged Calendar or Notebook apps for another thousand years if Apple could only solve the far more vexing software problems of AppleID unification when using iTunes and App Store, or the performance and reliability of the same. And yet these are the twin sides of the same systems design problem: the display layer surfacing or hiding the power within or, increasingly, lack thereof.

Yes, unlike any other company, we hold Apple to a different standard. We have for three decades. And we have been amply rewarded. If Apple’s winning streak is to continue, I hope Jony Ive never misplaces his Superman cape behind his Corinthian leather sofa…for he will need it.