Congestion, Content Buffering and Complexity

Just as a chain is only as strong as its weakest link, a network connection can only be as fast as its slowest link. An IP network is comprised of many connections and routers, and the path that each packet takes can be change during the duration of a connection (such as downloading a web page). Depending on congestion and routing the effective transfer rate can greatly vary during a connection. Usually the weakest link for many users is the speed of their ISP access service, and is therefore where congestion is most likely.

[As an aside there is also a lot going on under the hood, so to speak, to make internet data communication work, and can also bear on congestion. The ISPs and carriers employ many network design and operations staff, automated and manual network management, and all the hardware and software (and real estate) to keep things flowing smoothly. Data transport below the IP layer -- which can include, among other things, DSLAMs, multiplexors, ATM and MPLS -- and applications above the IP layer -- HTTP, SIP, RTSP,etc. -- are unconcerned with all of this network-layer stuff, and so I will ignore it all in this article.]

Congestion is a large topic which I will not attempt to cover in this article. What I do want to discuss is one aspect of congestion management, and that is network content caching. This is, in brief, the technique of reducing network congestion by placing content closer to the user. This is accomplished with a cache of files or other popular content that the network will redirect to when requests are received. Sometimes it is explicity accomplished with mirror sites, which you have likely encountered in the past, or implicitly in a manner that is transparent to the user.

I was reminded of this when I read this article in Ars Technica about buffering. There are a few things that bother me about this article, although it is generally pretty good, since it blends together topics such as congestion, buffering, latency and caching as if they were the same rather than the closely linked but separate items that they are. I don't want to dwell on the article too closely except to, I hope, add some clarity with the following observations:

1. Bit-km as the network loading metric: The object of mirror and cache sites is, in large part, to reduce the overall load on the internet across all of its component networks. If we exclude data compression -- the largest downloads are media files, which are already compressed -- we can only reduce network load by reduced the distance between the user and the content. If you are in Ottawa and you want to download a movie, the bit-km is lower if the content is cached, say, in Toronto rather than Los Angeles. The content is transferred to each cache once, and each local user doesn't tie up transcontinental network capacity.
   
2. Congestion has a time-frame: Imagine you are at the supermarket and you are looking for a cashier. If there is one free you would rush over there, avoid congestion for your transaction. If they're all busy, with other customers queued up at every cashier, you encounter congestion. Come back a few minutes later and you may find there is, again, a free cashier. This is an example of periodically high short-term congestion but low long-term congestion. The grocer's challenge is to engineer an acceptable amount of short-term congestion (long-term congestion is almost always bad) to optimize their economic outcome by balancing their costs and your continued patronage. Networks are similar, where there is some tolerance for short-term congestion as long as long-term congestion is kept under control.

When we talk about buffering in general it is about those line-ups, whether it be for cashiers or routing to the next link in a network data path. This is more about the impact on latency since caches and mirrors don't really affect congestion in the long run. The reason is that the network operators follow good engineering economics practice by only provisioning enough capacity to meet the demand; caches and mirrors reduce the overall demand and therefore the required amount of equipment and transmission capacity. Similarly, the grocer doesn't like cashiers sitting around doing nothing since it costs money, although as a customer you certainly prefer it that way. However all customers potentially benefit in both cases since with lower costs there can be lower prices.

Unfortunately all this efficiency has its own costs. Grocers have to do manage employee numbers and schedules against predicted -- never certain -- customer demand, just as network operators have to manage choice and placement of caches against predicted demand for that content. That complexity isn't free and therefore must be carefully assessed in every situation. All these systems add complexity to the network and create a need for specialized skills to manage the complexity. This is not only costly but also creates more failure modes. There have to be compelling cost reductions before taking on the risk of going down that path.

Against that complexity is the relative simplicity of adding more network capacity. This is a less-risky choice since it means doing more of the same thing: equipment, staff and processes. In addition, the costs are more predictable if, possibly, higher. Oftentimes throwing more capacity at the problem of both short-term and long-term congestion is the superior solution, at least until the pain of doing so becomes financially unacceptable.

If you want to come up against this first hand, try to sell a network operator on installing a new type of equipment into their network to solve the congestion problem. Should you succeed, congratulate yourself on achieving a monumentally difficult objective. More often you will fail but rarely turned away cold; the person you are selling to may know the potential benefit of what you're selling but will also know the risk to them (both personal and to their employer) of choosing unwisely. It is tempting to instead call up the Cisco sales rep and order a few more blades for those routers that are already running the network just fine.

User-transparent caching (or buffering, if you prefer) sounds good in theory but can be very costly in practice. Beware discussions of this topic that fail to mention complexity, cost versus alternatives, and reliability.

Skype Outage Changes Nothing

Skype suffered a wide-scale outage recently and it doesn't matter one bit since it will keep being used, will keep adding new features and will in fact keep growing. Does this seem counter-intuitive? If it does, it may be because of falling into the trap of comparing Skype (or similar services) to wire telephony service where reliability remains a key metric. The media, even the technology trade media, has largely fallen into that very trap. This is the wrong way to assess Skype's episodic woes.

A good indication of what I see going on is by comparing the Skype outage to this week's woes of the iPhone and its new year alarm glitch. Even though Apple at first responded in the same negligent manner as they did when their antenna problems came to light earlier last year -- by saying that the problem would self correct in a couple of days, which it reportedly did not -- I would imagine that no one expects that iPhone users are about to jump ship and buy Android phones. It just won't happen, nor should it.

Skype, like the iPhone, has a tremendous amount of consumer acceptance -- an emotional attachment -- that is very tolerant of occasional failures or even persistent problems. In their eyes, the benefits are so overwhelming that the negatives do nothing to change their views. Indeed, they will even rush to defend their favourite product or service when it comes under attack during these inevitable screw-ups.

One bit of link bait I read (which I won't link to!) claimed that Skype's outage could irreparably harm its increasing acceptance among business users. I and many of the people I work with are Skype users and although the outage was annoying I heard not one peep from my Skype contacts about how angered and upset they were about it. In fact no one said much of anything. The media, too, has gone quiet, or at least they merely moved on to the iPhone alarm problem as the next great headline fodder.

No one seriously compares Skype to their wired phone from Bell Canada or other telephone companies. They are instead seen as complementary, and also complementary in some respects to mobile phones. The reason is that Skype is really good at doing some important things that the humble and oh-so-reliable telephone cannot:

• Cheap: I can make long international calls (which I do for business) at zero cost.
• Features: Conference calls are a breeze to set up and manage, I can check on a person's availability at either a glance or with a quick text message, the audio quality is good, there's video and more features that I are nice to have even if I don't use them much or at all.
• Convenience: It works like a good-quality speakerphone or I can attach a headset to the PC for even better quality.

It works so well that I even use it for local business calls, despite the flat rate on the home phone. But what is more pertinent is that when Skype fails I still have several alternatives to reach people, including both real time communications and non-real time messaging. That is, I don't depend on Skype the way we once were utterly dependent on a monopoly-operated telephone system. In the case of the iPhone, I doubt that anyone this week couldn't similarly find several devices that would wake them up.

Conversely, Skype and its ilk are not about to render the phone company obsolete, or at least not so fast that the carrier industry can't adapt. They did it with wireless and the internet, and they could do it again, despite the distraction of cord cutters. However this is not to say that there is not a real threat that causes concern in some quarters.

Experts say companies like Skype operate in a legal grey area and that the notice is a warning to them not to grow too big or to challenge the state-owned telecoms...“This notice is actually protecting the telecoms' traditional voice services,” said Mr. Kan, [a director of China VoIP & Digital Telecom Inc., a company that has offered Internet phone services] who is also a professor at the Beijing University of Post and Telecommunications.

This is also why Skype and Google Voice had to scale back their mobile phone apps on carrier-locked smart phones, achieving an uneasy and unstable compromise between keeping users connected and protecting (to a degree) carrier voice revenue. It is unstable because the marginal cost of voice minutes is very low, and getting lower, and more people are realizing that simple truth.

Ofcom also found that cost, more than anything, determined how long people talk for and whether they prefer a landline or a mobile call.
...
Less talking does not necessarily mean less phone use. According to Nielsen the number of paid texts per subscriber has grown rapidly over the same period, recently surpassing 700 per month.
...
Skype, the internet phone service, is growing rapidly. In the first half of 2010 users racked up 95 billion minutes in voice and video calls.

For the present, Skype will continue to grow, as too will Apple and Google, among others. They will do so despite episodic glitches and outages since their perceived value is high. The carriers will continue to generate profits for their shareholders, but without the same enormous growth or even any growth at all. Of course the landscape could change sooner rather than later since it is a market ripe for massive disruption by the persistent march of technology.

Counting Watts and Technology Literacy

I wasn't surprised to read that many people have some difficulty appreciating the impact of their daily lives on the amount of electrical power they use. I my experience, not many people have an intuitive feel for this sort of thing. It was why, for example, in my article on solar insolation I avoided units like watts per square meter, choosing instead a more arbitrary unit, zenith-hours, that I hoped would resonate with the majority.

From the content of this blog you should not be surprised that I do not have this difficulty. With a strong technology background and a post-graduate science degree I have no trouble understanding machines, software and how many of the now-common and sophisticated devices work. There are also a great many things I don't understand, especially outside of technology; one person can't, or even want to know everything. I take care never to criticize another person's lack of understanding or their lack of interest in understanding.

Instead I have a curiosity about what most people actually do understand, or believe, about science and technology. I am often surprised by what I hear. It is also why I wrote an article about signal strength indicators on mobile phones after the iPhone "antennagate" issue went mainstream because of all the people I encountered that didn't understand the relationship between the indicator and phone performance yet wanted to know if or even why it mattered. That article continues to show up in my web stats as one of my most popular articles, which I'll take as a positive indication that there are people out there who do want to understand more about what makes modern technology tick.

As a side note, another misconception I have encountered, and may be widespread, is how GPS navigation devices work. Everyone seems to get that there is some sort of triangulation going on with one or more satellites that allows the device to determine its position. However there are people who believe that the satellites also download the maps and navigation information to the device. This is despite the fact that these units are clearly advertised with the world region modules and functions that they support. Then I discovered Google's mapping app on Android that does download the maps in real-time, except that it download over conventional terrestrial wireless networks, and is therefore pretty useless on many roads away from cities and major thoroughfares.

Getting back to the subject of counting watts, I have noticed a correlation between the perceived energy consumption of an electrical appliance and the appliance's position on dominance in the home or office. Apart from electrical heating -- which pretty well everyone understands is a major consumer of electrical power -- my non-scientific survey of people informs me that the big-screen television is the major electrical power expense in the home. I suppose it's because it's so present in most people's lives, it's big and it radiates a lot of light and sound. However that's just a supposition on my part because, when I do ask, I don't get any clear reason why they believe this.

Another blind spot is all appliances that produce heat: kettles, coffee makers, microwave ovens, ranges, refrigerators, and even clothes dryers, which are significant power consumers. Of course the totality of the energy they consume is limited by their non-continuous usage. The same goes for devices with large motors, including lawn mowers and washing machines. These are far from universal beliefs, it's just a pattern in what people tell me.

While this is a challenge for the government and the utilities it is perhaps less severe because of our cold climate. The reason is that all the waste heat from our appliance-filled homes heats our homes, in effect lowering the load on the furnace. Electrical heating often costs more than fossil fuel heating, but the differential isn't too large and therefore isn't a great concern most of the time, in my opinion. That is, between the months of October and April; summer is another matter since the additional heat is not only an unrecoverable cost but will, in homes with air conditioning, increase the electrical consumption even more in order to remove that heat.

Although the net impact is perhaps not exceptional, it does have an impact on public policy and electrical power generation. Summertime air conditioning is almost entirely done with electricity (there is no fossil fuel alternative as there is for home heating) which contributes to maximum power consumption occurring at that time of year. Lowering the heat from other appliances (even televisions!), including lighting, is particularly important in the summer in order to both reduce the load on power plants. While the fuel for the power generators can be reduced at off-peak periods, the generating plants themselves must be designed for peak usage, including additional capacity to deal with any temporary failures and downtime for maintenance. We pay for those capital costs since they are amortized over time with increased consumption-based rates.

China's Muscle Flexing

My reaction on reading Paul Krugman's article on China exploiting its globally-dominant position in rare-earth metals was an interesting one: laughter. While I have no especial reason to side with China in this or other trade disputes, I can only smile as they use tactics that other countries have used themselves for decades or even centuries, and in particular the US, which is unsurprisingly where Krugman is located.

The thing is, countries which have a strong military or economic position tend to exploit it to their own advantage. Trade is merely one aspect of that power relationship. The US used its power to do everything from opening up Japan to international trade in the 19-th century to creating the so-called banana republics in Central America to gain favorable access to tropical produce to imposing advantageous (to them) rules on exports of softwood lumber from Canada. This is simple power politics. Yet we are still able to maintain a positive overall impression of the US.

When a new power comes on the scene it is expected that the old powers will squirm at least a little bit. It is good for China, and us, that its per-capita wealth is growing and that the country is now in a position to trade with other countries on a more equal basis. Trade goes both ways, so while we import products from them, we also have a new, large export market. For Canada the situation is not too far different from the one with the US since in both cases the trade partner is big, aggressive and wealthy. Whether it is US petroleum money in Alberta's oil patch or China mulling over Saskatchewan's potash industry, the concerns for us are pretty equivalent.

In regards to the recent dust-up between Japan and China over sea rights, Krugman extends the threat to that which China could exercise over rare-earth metals:

On one side, the affair highlights the fecklessness of U.S. policy makers, who did nothing while an unreliable regime acquired a stranglehold on key materials. On the other side, the incident shows a Chinese government that is dangerously trigger-happy, willing to wage economic warfare on the slightest provocation.

I would suggest that China is no more trigger happy in these sort of dispute that the US has been in the past century; it is simply that China has some newly-acquired muscle and they are not afraid to use it. They are certainly being aggressive, but they are not going to war with anyone. In diplomatic circles this is called "sending a message". I say that it worked: China has succeeded in waking up some sleepyheads around the world, and in Washington in particular.

This sort of muscle flexing by China will continue, and it will be felt in all areas of trade and international business. This ranges from their wide-ranging mining and resource activities throughout Africa -- too often ignored by others -- to what I believe will be an assault on the the high-value end of the economic spectrum as evidenced by their growing pursuit of patents.

The bottom line is that China is a major world power that is increasingly in a position to flex its muscle. Like other powers that came before, they will naturally find that balance point between passiveness and aggressive expansionism since its optimizes their net benefit. That is, they refuse to be exploited as they have so often been in the past, nor will they overuse their military prowess to become an international pariah.

Their muscle-flexing causes perhaps more qualms than would another country since their government does not express the same fundamental themes of individual freedom and democracy. We could ourselves do better at this, yet China clearly does worse. There is good reason to believe that they will do better in the future. I am hopeful that China will in time become a more gentle giant than they are currently exhibiting as their growing pains create frictions with those they rub up against.

Police State Languages, and Businesses

In George Orwell's "1984" the totalitarian state government has created a new language, one based on ordinary English, called Newspeak. They are just beginning to introduce it to the population in the time period in which the novel's story takes place. It is not an optional language; it is positioned to become the state's official language and, over perhaps a generation, it will be the only language to be spoken.

Newspeak, as Orwell describes it, complete with examples, is restricted in both form and vocabulary. It is designed to make it difficult to say things which the state deems to be bad, and conversely making it easy to say the things of which they are likely to approve. Further, since we think in language, people will supposedly find it difficult to have thoughts that are contrary to the state's interests. If successful, the change in language would solidify the state's control over its citizens' minds and lives.

Despite some problems with the realization of such an outrageous idea, it is one that is a common topic in software engineering. Computer languages are strictly formal, with syntax and often semantics firmly based on propositional logic, with any deviation causing errors. Those errors must be corrected for a program to run. This fact makes computer language design quite similar to Orwell's Newspeak. One of the key questions a language designer must consider is whether to accommodate software designers' whims or creativity, or tie the structure down in a way that compels them to design software that meets the language designer's objectives.

This concept was first forcefully introduced to me by Robert Dewar, one of the creators of the Spitbol programming language (a version of Snobol) during a seminar he gave when he visited my university. With a nod to Orwell, he called computer languages that forced programmers to avoid a wide range of common, if sloppy practices as police state languages. The distinction is that in police state languages (which Spitbol most definitely was not) the objective was to make it difficult for the programmer to write "bad" software. Not impossible, just difficult, so that there was a greater likelihood that "good" software would be written.  Modern object oriented languages such as Java can be classified as police state languages since they rigourously enforce a data structure and flow of control that is inconvenient (and often pointless) to avoid.

Conversely, non-police state languages make it easy to write "good" programs and "bad" programs; it's up to the designer to make appropriate choices. Languages such as C are in this category. However, the dividing line between these police state languages and others is fuzzy, and the differences are sometimes subtle.

Beyond philosophical differences there are other advantages and disadvantages to both categories of computer languages that I won't get into here. Suffice it to say that most software designers use both at different times for different purposes.

There are ample analogous situations that can be found in all aspects of human endeavour. A bureaucratic company, and especially the civil services, is rife with police state modes of operation. It seems that the most trivial task requires the filling out of forms, the request for approval from one's manager (who may then seek the same from his or her manager), documenting how every hour of every day is spent, and so forth. These are organizations that make it very difficult to do bad things. They also suck the life out of their employees by destroying creativity, initiative and productivity.

Presumably these organizations are perfectly satisfied with the trade-off. Private sector organizations of this type (think of insurance companies and even, though to a lesser extent, the old Nortel) are vulnerable to competition or a rapid evolution in their markets.

Entrepreneurial companies that are aggressively growing are of the other category. These companies harness the energies of their employees in ways that are a joy to see and experience (everyone ought to have this opportunity at least once in their careers). A couple of local (to Ottawa) examples with which I am familiar were Mitel circa 1980 and Newbridge in its first decade. On the down side there is often total chaos, including frequent changes in direction, such as the trashing of one nascent product or market for something supposedly better or more promising, and no sense that the organization is able to function as a unified entity. The resulting inefficiencies are in one sense horrifying, except that the energies released can still result in prodigious results.

These non-police state businesses make it easy to do both good things and bad things, and there is a lot of individual discretion given to employees to make that choice. It is common in these companies to find few formal processes (just find someone with formal authority to sign a purchase order, as an example), nothing that could be properly called a budget, and often products that get shipped have surprising features that were never specified in advance.

As with computer languages there are market sectors that favour each category of business. Oftentimes a formerly free-wheeling organization will, when times get tough, adopt many of the practices of police state companies. Regrettably we rarely see companies move in the opposite direction, away from police-state practices. Instead they tend to die off.

For a manager in a free-wheeling company, it can be very challenging to maintain any sort of control over delivering to objectives. There is often the desire to institute controls. If you are ever in this situation I suggest avoid the easy out -- creating controls that prohibit bad behaviour -- and instead learn to better direct and maintain the team's energy level -- encourage and reward good behaviour. People aren't computers and trying to police behaviour to too great a degree, even with laudable objectives, is rarely the right choice. Leave police-state languages and practices to the computer language practitioners.

Patent Wars

Von Clausewitz famously wrote that "war is state policy by other means." We can go further and say that economics is war by other means; wealth is power that a country can use to fulfill state policy without mobilizing its military forces. The wealth of corporations, by use of corporate law and trade agreements, complements direct government action in these policy endeavours. In effect the state operates through the private sector by proxy much as two world powers have often funded other countries as proxies to fight their fights in furtherance of their own policies.

Patents and patent law are important parts of economic warfare. It is often a means to extract wealth from other countries by using international agreements to prevent cheaper competitive products that these countries would either import or manufacture themselves. Think of generic pharmaceuticals, genes, machinery, GMOs, among many others. It all works in a manner similar to the way Britain once forced its colonies to import products from British factories even though they could manufacture these themselves from the same raw materials that they exported to Britain. For example, Indian cotton and clothing.

Patents can only be used in this fashion by a country when its citizens and corporations have more and better patents than those countries it wishes to target. This ration can be improved by combining a low standard for granting patents and a judicial system that is loathe to question weak patents. By also allowing corporations a means to request import restrictions by the government on products that are alleged to violate patents, the power of domestic patents is increased. The US today does all of these things.

For this reason I was quite interested to read in The Economist that China is finally catching up to the US with its patent portfolio. The chart they show (from WIPO data) clearly demonstrates the trend. With China's rapid growth, large population and strong national desire to elevate their status among the great countries of the world, this is wholly unsurprising. A good example is their space program. Since the end of the cold war the US space program has drifted, having no clear objective for exploration beyond science. In contrast, China recently announced a new unmanned Moon mission that failed to make more than a small media ripple in this part of the world. The US and Russia are no longer one-upping each other to get to the Moon and beyond, but China is aggressively advancing their space program in a manner somewhat similar to what we saw in the 1960s and 1970s. They feel they have something to prove, and perhaps they do.

With respect to modernization and technological advancement, China is catching up, and patents are only one measure of their progress.

The question for America is not the influx of cheap but what do we do when cheap is gone and China starts to make the expensive stuff, too.

With let up in the trend, the quality and quantity of Chinese patents will pass Europe and the US very soon. As the innovation balance gradually shifts east there will be some profound impacts on the US and Europe, and their private sectors. As this article began, if patents are a battlefield the advantage will shift east as well. Patent law, which currently favours the US due to their larger and stronger portfolio, will, perhaps as soon as 2020, begin to benefit the China more often. Since old patents expire and the majority of new patents will belong to Chinese companies, the winner of more and more patent lawsuits will be won by Chinese companies.

When this happens I think we may finally see patent reform. At first in the US and then, if they negotiate wisely, internationally. If the bar is not set higher so that weak patents cannot be used as weapons in the court, US companies will suffer. Until this happens I suspect that Congress will not change course. However, this is a large ship that changes course only very slowly. Even so it is unlikely to change the outcome, especially so if the US continues to turn out fewer engineers and scientists, or just at a slower rate of growth than the competition.

RIM and Cisco Pursue Lateral Expansion

One of the more fascinating things I like to watch is large, established technology companies attempting to branch out into lines of business in which they have no history. I prefer to do this as a spectator, at some remove, since it so often goes so spectacularly wrong. Acquiring or merging with a company is never for the faint of heart even when both operate in the same space and have truly complementary products and services. I still remember how Nortel stumbled from one ill-conceived acquisition to another in management's attempt to transform Nortel into an "IP company".

There are two recent, though still rumoured and unconfirmed examples that caught my attention: RIM is allegedly in talks to acquire a mobile advertising firm, Millenial Media, and Cisco may be pursuing Skype. The former rumour is better substantiated in the press, and the story also passes the credibility test since RIM's key competitors -- Apple and Google -- are moving aggressively into mobile ads as a potential future revenue source. It is a stretch for Apple, but perhaps even more so for RIM which does not yet have much consumer sector success in either its products or services, especially expertise in consumer marketing and design. I was nevertheless amused to hear this rumour the day after posting my article describing RIM's challenge to finding a new recurring revenue source.

In Cisco's case the driver behind their move is less clear to me. It is no secret that they've wanted to get into people's homes for some time now, and that supposedly drove their acquisition of Linksys several years ago. Regardless of how successful that move may have been to revenue growth or profitability, it would be a stretch to claim that their brand awareness in the consumer space is much improved. Skype would change that situation dramatically. Of course Cisco has had a presence in consumer VoIP services for some time, starting years ago when their ATA products were distributed by Vonage and others to couple analogue phones to home broadband services, but Skype would be a big, big step up from that.

Unlike RIM, Cisco has a track record of making many successful acquisitions, and doing so at a rapid pace. They seem to have the practice down to a fine art. However, integrating Skype into their operations will be less routine because of its size and, most importantly, Skype's business being so very different from what Cisco now does. This is no small matter since it is one that has ground down many companies. Can Cisco do better? Perhaps. They should at least be able to do better at it than Ebay managed.

The reasons why many companies fail at this sort of lateral expansion into new businesses and market sectors are many. Unless there is a good strategic vision to justify the acquisition that is also executed exceptionally well, the acquisition usually just disrupts the operations and successes of both parties. In the end the company may once again be sold, passed onto a hopefully better parent, or kept and operated at arm's length as a separate subsidiary. The latter is fine if that is the original intent, but if not then it can be a distraction to senior management and confusing to shareholders even if the acquired company continues to grow and be profitable.

I would very much like to hear what Cisco would propose to do with Skype if they are really interested in the company, to somehow integrate it into their operations for some synergies (that was Ebay's stated reason, though they failed to even convince anyone that it made any sense), or simple as a marketing tool to raise their profile with consumers and drive sales of their current and future consumer products. That's the best justification I can come up with right now, and it is one that I do not find compelling. I eagerly await further news regarding this rumour.

Perils of Protecting Privacy While Browsing

The software and services we all use every day, whether stand-alone or over the internet, are incredibly intricate and complex. It is one of those truths of software engineering that defects (or bugs if you prefer) grow in number faster, not in proportion, to the number of lines of software code. Windows often take the brunt of the bad public relations when things go wrong with our computers, probably because (for most people) it's right in front of them when trouble occurs. Yet the problems are to be found everywhere. No software vendor or open-source project is immune to exploits. Even when trouble is not due to a defect but some obscure though intended function, it is perceived by users as a defect.

Browsers are a case in point. Due to their complexity, interconnectedness with all sorts of local software, web sites and network services, and their rapid evolution in the face of competition and changing standards, they are frequently the vectors of malicious attacks. By attack I mean more than viruses and other malware: anything that flies under the radar to track, mislead or pry into my activities, online or offline, is an attack against my privacy. This includes cookies, browser history, and even remote tracking based on IP address.

It was therefore of no small interest to me to read about the latest exploits against the private browsing feature that most browsers now support. I can't say that I'm surprised to hear this news. The so-called sandbox that private browsing features create must by their nature be quite complex in how they must pick and choose communications paths to connect with or block among the multitude that all modern browsers contains. Recall what I said above about defects and complexity and you can see why I am not surprised.

For a long time now I have followed my own path to security and privacy, choosing to use many browsers. Each is set up and used to accomplish a specific function. My primary browser is Firefox, and I tend to use it to visit web sites I generally trust. Yet I use an assortment of plug-ins to protect my privacy, including an ad blocker, a cookie manager and a Shockwave Flash cookie manager. (You know about Flash cookies, don't you?) I use a completely different browser to visit untrusted and unknown web sites the first time, or even trusted web sites if I want to disrupt how they track my activities. Being human I do make the occasional bad decision when I'm in a hurry, and I have paid the price.

My reason for going this route -- although I have experimented with private browsing features -- is that I put a premium on privacy and security (that's one reason why this blog is anonymous). I expect private browsing to be a fragile and delicate thing due to its complexity. I therefore choose to address the problem with a sledgehammer rather than a scalpel; that is, brute force over finesse. On the second browser I have Javascript disabled by default and at the end of every session all private data is deleted. I deal with Flash cookies for all browsers via the Firefox plug-in BetterPrivacy. When I use that browser as my own little sandbox I do not impede cookies or other trackers. It isn't necessary, nor is private browsing, since it all gets flushed when I exit.

Countering this zealous slash-and-burn approach is the question of whether privacy itself is passing into history as we pass into a future of no privacy and complete transparency. Google CEO Eric Schmidt seems to lean in this direction, but then his company has a vested interest in following you around. Others without these commercial interests have also thought about the issue and come to the same conclusion. Some like author David Brin go so far as to argue that this is a good thing so get used to it and turn it to your advantage.

I am not ready to get used to it and perhaps I never will. To me privacy and anonymity occupy an important part in my life so I take steps to make it happen. Browsing privacy is just one piece of the puzzle. I will continue on this path even as it gets more treacherous as technology moves forward. There are interesting times ahead of us as privacy rights get increasingly mangled and redefined.

RIM's Hold on the Enterprise Mobile Phone Market

There is an enormous amount of angst among some Canadians regarding the future prospects of RIM. It seems that since the demise of Nortel they have been burdened with the title of Great Canadian Technology Hope. The worry goes further of course, since due to their size they have a large investor base and there are legitimate concerns with its ability to compete against technology darlings Apple and Google with their iPhone and Android mobile phones and eco-systems.

RIM does have a loyal market for Blackberry, and that market is growing. It's their market share that is at risk. Blackberry, iPhone and Android are each growing their market as they jockey for appeal to the disparate segments of the smart phone market. Blackberry primarily appeals to the enterprise market, and its attempts to become interesting to consumers have not always gone well.

But kids don’t want Blackberries. They want cheap phones that run instant messaging, SMS, and email. Need a keyboard? Get a MyTouch Slide. Want a big screen? Maybe an EVO 4G.

Perhaps the Torch and OS6 will get them there, or maybe not, but they will press on regardless. Similarly, Apple strives to reach out to enterprise market from the consumer market where they currently dominate so completely. They, too, have their challenges. Then there is Android which has a more diffuse base, though slanted more toward consumers; Google appears to be leaving it to the device manufacturers to decide which markets they want to address while they focus more on total numbers rather than who those users are.

Something I do feel confident predicting is that, ultimately, the smart phone market will be the same as the mobile device market, which is the expected end result of decreasing component prices and lower cost of fewer standard platforms; every phone will be a smart phone, and many mobile devices (tablets, netbooks, etc.) will utilize the same platforms. There is room in that future market for more than one platform, and indeed more than two, but perhaps not more than three (apart from niches). But for the present let's look at what it means that different platform appeal to different market segments.

Business users love their Blackberrys and they love their Blackberry email. I am not a Blackberry user so I always find it both fascinating and amusing to watch people in meetings constantly glancing at their devices. When they do pick up a message and respond, they do it with the speed and finesse of long experience. It is easy to see why even now when the platform is quite old as technologies go that Blackberry received the moniker Crackberry.

Another funny thing about dedicated Blackberry users, in particular those in the technology circles in which I operate, they love playing around with iPhones and Android phones. The touch screens, the colours, the apps, well most everything about them appeals to the engineer turned executive. But once they're done playing with a colleague's phone they feel a vibration or hear a quiet ping and they drop it, now completely forgotten, to check the latest incoming message on their Blackberry. It's almost uncanny how often I've witnessed this occur. These are not people that will easily abandon RIM products and services. To them, the newer generation smart phones are eye candy and nothing more.

Another aspect to this Blackberry attachment in the enterprise is its hold on the companies whose employees need email on the go, and its bizarre impact on their employees. Many times I have sat down to lunch or a coffee to discuss business with someone who will first pull their phones out and place on the table. This is done not only to watch for messages but because, light as they are, it is still a relief to detach the weight from their clothing when they can. However it's what often happens next that I find so interesting. After the Blackberry is on the table they pull out another phone and similarly place it on the table and, if you can believe, some will even pull out a third phone and do the same.

When this first starting happening I would stare at the person opposite with the unspoken question hanging in the air between us. They would then feel compelled to explain. The Blackberry is used only for email, not phone calls, since that is the mobile device approved by their IT department and integrated with the company's email system. While it can be used as a phone, and may indeed have a contract and number assigned, the person does not use it that way. Instead they have another phone for business calls. These are frequently sales people with a large network of contacts who feel a need to keep the same phone and number when they more among employers. To them it is vital that they keep their phone number sacrosanct so that everyone can always find them, and they often can't do this with the corporate Blackberry or, they worry, they may not be able to get their number back when they next change jobs.

The third phone, when there is one, is something like an iPhone which they use to play with and perhaps to stay in contact with friends and family. This isn't strictly necessary since smart phones can each do so much but they do it anyway. Perhaps it's the keyboard on the business phone (sometimes their first two phones are both Blackberrys) or perhaps it's the the social networking, apps and music on the personal phone that drives them to act this way. If you've ever watched a Blackberry user struggle to do something on their phones other than email you'll know why (at least pre-Torch and OS6) users do currently need another device for entertainment and general internet use. While I don't have a Blackberry I do carry two phone: a pretty basic feature phone that is just a phone, and an Android phone for internet, email, apps and much more.

The thing is that as smart phones become the norm and they all more-or-less support the same functionality there will be a change among enterprise users as they make use of the hardware varieties and new cloud phone services like Google Voice to eliminate the current ridiculous situation by reducing back down to one phone. That phone will have to support multiple numbers and possibly phone clients (including VoIP), and sport a keyboard for those unwilling to undertake the learning curve to become proficient on a soft keyboard. The big question is, which phone will that be?

Ponder that while RIM tries once more to make a splash with consumers. Will enterprise users drop their second and third phones if they can? Will enterprise IT departments loosen their choke hold on employees' phone and service choices, and in particular will they ever become truly comfortable with iPhone or Android devices? Until these questions are suitably resolved, many business people will continue to own more than one phone and dominance in the enterprise market will continue to be held by RIM.

Blackberry and RIM are a long, long way from becoming irrelevant. I will also say that the current concerns of some governments over Blackberry email security will not only not harm RIM but will help them; it's great (and free) advertising that their encryption and security perimeter are so strong that even governments can't break it. Users and their companies get the warm fuzzies just thinking about that fact.

Meaning of Signal Strength Bars

Apple's antennagate controversy is now growing old and a bit stale. The company has come out with a half-hearted response to dealing with the problem, but it appears increasingly likely that this will prove sufficiently satisfying to their customers, especially since most have not experienced the problem of fringe-area performance. There has even been some talk about the antenna being the reason for delaying release of the white iPhone 4, or then again maybe not.

Although I've already touched on this issue, there is one aspect that I think is worthy of another blog post. It comes about from a few subsequent discussions I've had with people, including iPhone 4 users with the product in question. This has to do with a fundamental misunderstanding of how cell phones are internally engineered. I often take it for granted that in this modern age when it seems everyone is conversant with the high-tech products and services that they use every day, that there is also some basic understanding of that technology. This is frequently untrue. Rather than seeing the cell phone for what it is, internally -- a collection of modular building blocks that are essentially independent and communicate over interfaces -- some (most?) people see the cell phone as a monolithic device. That is the impression I get as to why there is still so much confusion between the dependence of the iPhone 4's antenna performance on how it's held and the number of bars on the signal strength display.

But touching the hot spot doesn’t always ruin the call, even if it lowers the number of bars. In several cases, when I was already on a call with three or four bars showing, I deliberately covered the hot spot with my hand, and the call continued normally, strong and clear, even though the bars dropped to one or two.

On that basis I decided to break down the problem into its component parts so that the relationship is more clearly illustrated, and in particular why the number of bars of signal that is displayed can be so distinct from fringe-area performance. Hopefully someone will benefit from this. To those who do understand the technology well, please forgive me for any errors due to my coarse description of the technology details.

First however, let's look more closely at weak-signal performance, where the received signal strength is close to the minimum for a usable signal. This is determined by the signal's field strength -- which is independent of the phone -- and the phone's antenna and radio module. As can be seen in the diagram, the gray area where performance is marginal is wider for the now-obsolete analogue technology (such as AMPS) than for digital. Degradation as an analogue signal weakens is more gradual, and is manifested with increasing noise and distortion until the other person's voice becomes completely unintelligible, and finally descends into silence and call termination.

With digital, degradation may not be noticed until the bit error rate is high enough to cause drop-outs. From there it takes only a little less signal to reach the condition where the bit stream cannot be decoded, resulting in silence and then call termination. In other words, the gray area of received signal strength where the signal is distorted or subject to drop-outs is narrower for modern digital phone transmission. As mentioned in my earlier article, this is one reason why the number of bars can be uncorrelated with reception quality: at any level above the minimum signal strength to achieve a low bit error rate, digital reception is pretty much perfect.

With that out of the way, let's return to the above diagram. The radio module is a self-contained unit, with reception dependent on no other component other than than the antenna system. If the antenna efficiency is impaired (or the signal from the carrier's base station is reduced by terrain or obstructions, or there is interference from another source) the signal that the receiver has to work with is reduced. There are differences among phone in their antenna designs and placement, and some variation in the performance of radio modules from component manufacturers, but for any one phone there are no other factors we need to be concerned about. The antenna plugs in one end, and audio in and out plugs in the other end, plus various control lines and power.

One thing the radio module produces is an indication of signal level (using an internal signal sampler) that can be read and used by other modules. This is typically to be interpreted on a relative scale where zero is approximately the level where communication with the network base station (or tower, if you like) is marginal or lost.  On the Nexus One phone, the relative scale goes from 0 to 31, where 0 is referenced to -113 dBm and 31 is -51 dBm. Below and above these signal strengths will produce 0 and 31, respectively. On other phones the granularity may be less (less than 32 values) and the interval between values may be non-constant (it's a constant 2 dB on Nexus One). However, these are at best nominal values and could vary, perhaps even quite a lot, since there is no real technical need to ensure accuracy.

Whatever the granularity and range, these values are mapped by software to something that is displayed for the convenience of the phone's user. The mapping can be linear (as shown) or any function at all that maps from the signal strength value to the indicator icon. The important part here is that the mapping is arbitrary. I've shown it as mapping the signal strength to one of 5 different displays -- 0 to 4 bars on the screen icon -- because that is a common format.

When Apple talked of making a software change, it was this mapping function they were discussing; they proposed changing one arbitrary mapping to another arbitrary mapping, one where the number of bars would be higher for lower signal strength values. If by chance this wasn't clear before, you should now be able to see that this mapping function has nothing whatsoever to do with the performance of the antenna system and radio module; the mapping function does not impact reception quality so modifying it does not solve the iPhone 4 antenna problem.

Of course changing the mapping function so that it shows more bars at lower signal strengths doesn't hurt. It may even be reassuring to those who worry that a call will be interrupted, or never begun, when there is just one bar displayed. This really isn't a terrible idea since, as described earlier and in the previous article, performance is probably just fine at that signal strength on a digital network so why not remove the source of worry. The thing Apple should not do is claim that this change solves the problem, since it clearly does no such thing.

There is also the matter of the phone's transmission performance since it, too, is equally dependent on the antenna; antennas are in general reciprocal in receive and transmit usage, so a reduction of antenna performance affects transmission (from the phone to the base station/tower) equally. Since that process is a story by itself, I'll stop here, having covered the reception issue which is the larger problem.

Smart Phones and Signal Strength

Along with rest of the world, I have been looking on in stunned disbelief at how poorly Apple is dealing with the iPhone antenna problem. I am less troubled by the problem itself -- although it is not inconsequential -- since it has the smell of an unfortunate engineering compromise.

As this article indicates, signal attenuation is hardly unique to the iPhone. I can confirm that the Google (HTC) Nexus One does indeed have the same problem, as the article shows in its comparison. In a fringe reception area the problem can be severe, leading to the reported lost calls by iPhone users. Most urban users with an long-established carrier are unlikely to see the problem, or only intermittently when travelling. In my case the Nexus One is registered on Wind Mobile, and their coverage is poor, seemingly relying on fewer base stations ("towers") than Bell and Rogers to cover the city. Hopefully they are working on filling in those holes, but for now I am in a fringe area where the signal loss from simply holding the phone is catastrophic.

Fringe area performance is so noticable since with digital transmission technology there is a small signal strength interval between no signal and a perfectly clear signal. This wasn't the case back in the days of analogue technology (e.g. AMPS), where the degradation was more gradual rather than the modern phenomenon of sudden transitions between signal loss and acquisition. That is, you may not realize you are in a fringe area, and therefore prone to a dropped call, until it happens. This relates to the number of signal bars that are shown, and partly excuses Apple's reporting of more bars, since with digital transmission it isn't that big a deal provided there is enough signal to work with. In other words, if the phone works, for most people the number of bars is superfluous information.

Of course phone users do care about how many bars they get since it is one of the few ways they can judge how well their carrier is performing. In this respect, Apple's move to increase the number of bars reported (which has nothing at all to do with the phone's radio performance) annoys their customers. While not as serious as the speedometers in some early generation Japanese cars (early to mid-1970's Toyota Celicas are models that comes to mind) that consistently, and deliberately, read 10% higher than the true speed. The only purpose for cars and phones to do this is to mislead customers to an unwarranted favourable impression of the product.

CNET, for instance, ran some tests, and suggested that Apple might simply be juicing the signal display to make it look like its phone was getting good reception.

What I mentioned about an engineering compromise is worth comment. The insides of these big-screen smart phones are packed wall-to-wall with a lot of components. For best performance the antennas need to kept away from metal (many components and even the case) and coupling to or transmission through semiconducting objects (human body). The true situation is more complex than this short description, but it is enough to say that, under the many other constraints in place, placing the antennas around the outer edge of the phone is not unexpected. Avoiding coupling to the hand is best accomplished with distance, where even a few millimeters can make a difference, such as that provided by the rubber bumpers that Apple sells (I'm not aware of anything similar for the Nexus One).

Was Apple aware of this problem when they released the phone? I believe they were, and indeed it would have been extremely difficult for them not to know. But when it comes to large corporations and diverse priorities between engineering, marketing and management, it should surprise no one that engineering concerns about antenna performance would be acknowledged and set aside in favour of getting the phone into the market. After all, few people pay the slightest attention to the radio performance of a cell phone, preferring to focus on features such as camera megapixels, colour and song capacity, and the vendors respond accordingly. There are companies out there that do have a good reputation for radio performance: Motorola and even Nokia are examples that come to mind, but even they are inconsistent across the years and product lines so don't rush to them too quickly if that is what you want.

Apple will survive this road bump, probably even if they continue to obfuscate and mislead in their public relations. The brand is simply too strong and the majority of users won't even see the problem. However, a little dose of honestly wouldn't hurt them either.

Infected

I am pretty rigourous when it comes to safe computing as I travel the internet. I keep special-purpose sandboxes with which I can safely visit unfamiliar web sites and inspect downloaded files. Never do I click on strange links, even going so far as to ensure the link really points to where it says it points, avoided URL shorteners and using a Javascript-disabled browser for peculiar links that I absolutely need to open. Downloads and attachments are at first handled like toxic waste until their contents can be reasonably assured to be exploit-free. Where possible I also avoid software that is most-frequently targeted by hackers and malware.

Imaging my surprise when, recently, several bit of malware, including a backdoor/trojan, infected one of my Windows machines. It was entirely my own fault since, in a rush to get a job done, I opened an unfamiliar site without taking any precautions.

I knew immediately that something had gone terribly wrong, although I wasn't quite certain what had actually occurred. It involved Adobe software and I had gotten complacent about compatibility errors because I was not current to the latest version of Acrobat Reader. It was about an hour later that I knew what had happened and that my day was about to be ruined. Afterward, when I reverse engineered the sequence of events that led to the infection, I was able to discern just how the exploit worked. It's interesting in a way so let me take you on a brief tour of the steps.

1. The server with the news story I wanted to read had been compromised to serve files that contained the malware.
2. The infected file was given a .pdf extension so that it called up the Acrobat Reader browser plug-in to process it.
3. The old version of Acrobat Reader has a security hole that allows remote execution of unvetted code.
4. Several pieces of malware were installed using this security hole, including at least one that it subsequently downloaded from a server that hosted malware.
5. The malware started running and phoning home, doing who knows what. I stopped this immediately by stopping the processes and disconnecting from the internet. Unfortunately, the malware kept starting up again. This is pretty typical.

The malware was clumsily built, which made it easier to deal with using only manual techniques. First, it corrupted Acrobat Reader to the extent that it would not run at all, except to restart two malware processes. I would have expected it to not call such blatant attention to itself. Second, one bit of malware attempted to run as System but did not have a digital signature that would pass Microsoft's most basic security checks. This is really, really amateurish.

It took a bit of digging, but it was possible to find and excise most of the malware component using nothing other than file searches. Where I hit a wall was with inside that pit of obscurity, the Windows registry. I knew the malware was still hiding in there, able to reactivate itself, but I could do little with the regedit tool without special knowledge. This step required one of those freeware tools for finding and surgically removing malware from the registry.

One tool that is absolutely useless in this regard is Microsoft's own MSRT software (Malicious Software Removal Tool). Never in my experience has MSRT found malware on my computers; it can identity only a small number of malware variants, and seemingly never those that are actually circulating.

In any event, the malware tool I selected had no trouble finding and repairing every infected registry entry, and even located an infected file that my own efforts had missed. That PC once again has a clean bill of health and is humming along ever so quietly, undisturbed by uninvited software.

It just goes to show that even someone with software expertise and safe computing practices can fall victim to malware. The many benefits of the internet are not enjoyed without continued vigilance against its bad actors.

Risks of Flying

Those in Ottawa are likely familiar with the incident last week when a landing airplane overran the runway at the local airport. Luckily there were no serious injuries. This brings to mind a similar incident where I was a passenger on an aircraft that did something eerily similar. It is one of those things that, if you fly often enough -- and I've done a lot of business travel over the years -- is bound to happen.

In the flight I was on the landing hazard was a low ceiling combined with a sudden snow squall. I was in a window seat so I got a pretty good view of the landing as it transpired. I remember the long descent through an unbroken cloud deck, knowing that we had to be getting close to the ground, when we did break through the low ceiling perhaps only a few hundred feet above the ground. The descent continued through the heavy snow fall for a few seconds until we reached the runway lights. Alongside a few dim, snow-covered lights was a flat expanse of white where the runway had to be.

At this point our descent slowed for a moment -- it could be that the pilot was assessing the landing conditions. Then down we went, kicking up thick clouds of snow as be bounced down the runway. The pilots were braking slowly, careful not to lose control as the plane tried to gyrate on the slippery surface. It brought to mind something that a pilot once told me about one of the difficulties of flying a two-engine versus a single-engine plane: you have to finely adjust the thrust of each engine to avoid spinning the craft on a slippery surface. That's one reason why piloting a single-engine airplane requires less training. I suppose modern avionics must simplify this task, but this particular incident occurred many years ago.

I was familiar enough with the airport to know that we were getting near the end of the runway and our speed was still much too high to turn. Then we came to the very last taxiway at the very end of the runway and the pilot made the attempt. It didn't work. The plane made a half turn and proceeded to slide sideways into the field in the angle between the runway and taxiway.

At this point it is worth mentioning just how tough modern aircraft are built. To an unpracticed eye they seem to be terribly fragile machines that are at the whim of the forces of nature. In actuality, the undercarriage of an aircraft, including the wing structure, are probably the strongest part of the whole. After all, we have to remember that the entire weight of the aircraft and dynamic loading due to turbulence are supported in flight by the wings. The small motions of the wing tips look misleadingly ominous while in the air, but really say nothing about the total assembly's strength.

As we slide sideways into the snow and ice-covered field there was a violent shuddering, yet we stayed almost perfectly level. Some of the passengers had just enough time to say "oh!" or to suck air into their lungs in surprise, and then the plane stopped. We weren't about to go anywhere fast, but there were no injuries. A few people wondered what had actually happened since, like many people who fly a lot, they weren't paying any attention during the landing.

The rest of the story is boring since it relates to security and bureaucracy, and much waiting around for things to happen, so I'll skip all of that. We turned out to be the last flight of the evening as the airport authorities, belatedly, shut down the airport and diverted incoming traffic elsewhere until they could get the runways in decent shape. In the media, it is said that "if it bleeds, it leads", so, absent any bleeding or even bruises, the incident got only minor news coverage. It gets one wondering how many more of these minor incidents are never noticed because of the lack of media interest.

Happily, that was the only borderline serious incident I've been directly involved in, and it did not put me off flying a bit. We do tend to remember the few horror stories about flying, but not the vastly greater number of flights where everything ran smoothly. It remains a great way to travel, if you can overlook the tedium of getting to and from the airport and navigating security measures.

Gmail Is Just Email

I hope this story is true. Out of necessity I have a few Gmail accounts, and apart from the large storage offered I have always hated it. It isn't that it doesn't work well (other than the occasional outage). It's that they went with an interesting set of ideas to reinvent the email user experience (UX) that I find so non-intuitive that this software geek finds it extraordinary irritating.

Feature like conversations (per the referenced article by Henry Blodget), tags and other peculiarities are very interesting ideas. They are also just not what most users expect or, in many cases, will accept. They gave way some time back by offering folders -- just like other email systems -- which I found to be a great relief. Google seemed to be so enamoured of search that they expected the insertion of a few key attributes (tags and keywords) and auto-linking of messages (conversations, or threads) would allow users to find what they needed via a search box.

It was all very interesting and worth the experiment, but apart from what seems to be a minority of enthusiasts for this approach, Gmail's peculiarities have been mostly tolerated, not appreciated. Google is slow if not resistant to user feedback, and I wondered if they would ever change Gmail's UX any further.

Even if this latest change bruises a few egos among their staff, it is worthwhile. I applaud them for doing it since it will most likely benefit them (and me). As they found with their mass storage offer, there are ways to differentiate their services other than adopting innovative, though non-intuitive, UX strategies.

Android - Why Google Needs It To Succeed

If you've followed this blog for a while you'll know that I am involved with Android application development. This means I have a vested interest in seeing Android succeed. It doesn't mean that I will be ruined if Android fails or falls short of iPhone's appeal, just that as matters currently stand I benefit from Android's continued market acceptance.

Regardless of the my situation, I have always been puzzled by Google's intentions with regard to Android. Their attention (and resources allocated) to the Android Market and to their OHA partners is exceptionally poor. I would have thought that they would at least give the Android business more personal (human) attention than they do to the bulk of their products and services -- Google typically excludes any human-operated aspect to their support systems, relying instead (it seems) on random sampling of feedback and automated, statistically-driven response systems.

It is natural to question just how committed Google is to Android. They have not even adequately explained how HTML5 and their Chrome OS will be positioned versus Android and native Android applications in their overall market strategy. They play Android close to the chest, just as they do with everything else they do. That's their choice. Ultimately, even for a company as technology driven as Google is, there has to be money in it.

Lots of delivered and rumoured phones, tablets and other devices certainly demonstrates that Android has gained broad acceptance by carriers and device manufacturers. Since the platform is available at no charge, this is not how Google makes money from Android. It's been said that the money would flow from the Google experience applications and services that are typically bundled with Android, but this is optional and therefore uncertain, as has already been seen in several devices. Again, Android on its own is not a guaranteed revenue stream for Google.

I suspect the actual business is one of strategic positioning versus the alternative platforms, especially Windows and iPhone. Without a presence in the mobile device market -- Google does not build hardware -- these competitors can lock Google out of the mobile ad market. This comes about in two ways: mobile search and in-app advertising.

...Google prefers to have mobile users access the mobile web instead of have them locked up within smartphone applications. Users tend to use the Google search engine more if they’re surfing the web, which brings Google more cash.

Since Android had to be open with regard to applications and services to gain acceptance with carriers and device vendors, there is risk that Android adopters would sign deals with other search companies such as Microsoft's Bing and other ad networks such as AdMob. However, and I believe this is the key point, Google is at even more risk of losing search and app presence on iPhone and Windows. This is why, for example, the niche mobile advertising marketers are being acquired by Google, Apple and the other large players.

Google is also threatened by Apple's move into in-app advertising with iAd. Since Apple makes its money on selling hardware but not on software or content (these drive hardware sales), they are happy to create strong incentives to app developers and advertisers to use iAd.

If my analysis is right, Android is Google's opportunity -- although an uncertain one -- to protect its ad business for search and apps when its two competitors seek to lock them out on the other mobile device platforms. With the mobile web and apps taking an increasing share of total time spent by users on the internet, this is no small concern, even if there remains a real possibility that native apps will eventually be supplanted by HTML5-driven web apps.

I'll be watching announcements coming out from Google this week during their I/O conference to see if there are any that indicate their future direction on apps, Android, Chrome OS and mobile search.

Palm and HP

In the week since HP announced that it would acquire Palm for $5.70 per share there has been rampant speculation in the trade media as to what it all means. Long-suffering Palm shareholders have stars in their eyes, and are even holding out hope for a better deal, which is reflected in the persistence of trades above the offer price. Typically the share price for a deal expected to close as-is would hover ~2% below the offer price, or about $5.55, establishing equilibrium between sellers looking for an early exit and arbitrageurs willing to wait for the deal to formally close. A higher price is unlikely since there is no obvious value of Palm's assets to another buyer at a higher price. Which brings us to wonder how it is that HP thinks that Palm is worth $1.2B.

It's no surprise that HP wants into the smart phone market since it is an easy reach from their PC business, which focuses on computing hardware, and was widely expected ever since their largest competitor, Dell, made the move. After all, if consumers are beginning to prefer smart phones and tablets over traditional PCs and netbooks, the manufacturers of those devices must follow the money or lose revenue. Except that where Dell chose Android as the platform for their own devices, HP is opting for webOS and devices which, currently, only run webOS.

It is true, as some commentators have said, that this approach differentiates them from Dell, Apple, HTC, Nokia, Blackberry and other smart phone vendors. Unfortunately they have tied up with the platform most likely to lose. While this is something that many have predicted (including me), it is also apparent in Palm's steep revenue decline, which reflects low interest from consumers, carriers and app developers. HP said that they are serious about webOS and plan to invest further in the platform. They have the resources to stick it out for years if they are truly committed, but will webOS see a turnaround now that HP is behind it?

Success can be measured. Key indicators to watch for in the next quarter are a halt to the decline in unit sales and positive indications of interest from carriers beyond Sprint, and especially the largest carriers in the US and elsewhere. They won't close deals that soon, but if they can't demonstrate real interest then there is reason to doubt. Unfortunately for HP, I very much doubt that consumers and app developers will be impressed by HP's purchase of Palm. Consumers may respond to a new and extensive marketing campaign, which is unlikely to occur until the deal closes later this year. Attracting more app developers will be delayed until there are hard numbers that show webOS device sales are trending upward.

Yet there is still the matter of all those smart phone platforms in the market, among which webOS is the weakest of the bunch. Assuming that HP wants to keep webOS proprietary -- unlike Android, Symbian and some others -- they need to start pushing out new and compelling devices soon. But, again, they can't do this effectively until after the deal closes. It is conceivable that in the interim Palm could license the software to HP so that HP can get to work immediately, although this would be irregular since the deal itself will have to undergo some (routine) regulatory scrutiny and could imperil Palm's ability to attract and respond to better offers. This is a matter of shareholder interest that cannot be easily dismissed.

Apart from webOS, the other major assets in the deal are Palm's patent portfolio and the hardware devices. The devices are probably not worth a great deal in comparison to webOS and the patents since HP will have its own device plans and, besides, devices age rapidly in the market and all smart phones utilize components from pretty much the same suppliers: the vendor pretty much just assembles them, modifies the software to run on the device, and gets the regulator to certify the final product for sale. Look at the bills of materials for iPhone and any Android phone and they are noteworthy for their striking similarity.

The value of Palm's patents is debatable. Since I doubt that HP intends to license any of it to others or to sue other vendors, the patents' value is most likely to be defensive: to negotiate cross-licensing deals and to shield against suits from others. That is not inconsequential since these lawsuits can result is heavy costs and awards, easily running into the hundreds of millions of dollars, each. This implies that even if HP were to dump webOS and go with Android, purchasing Palm might still be justified in their move into the smart phone business.

Getting back to differentiation, I would say that it is not so much webOS itself that provides this to HP but the Palm team that produced the device and its user interface. In comparison, Dell's and Acer's Android devices appear (at least so far) to be pretty generic Android. This is very unlike HTC, Motorola and even Samsung Android phones. The differences is that PC companies do not have the experience with developing custom user experiences, instead relying heavily on Windows. However one chooses to value the Palm team, this does give them a leg up on their traditional competitors. How this will stand them in the broader smart phone market remains to be seen.

After all this rambling, I still find it difficult to come to any conclusion that I find satisfying. HP has a difficult path ahead of it, where success is strongly tied to their unproven ability to nurture webOS into a contender rather than a might-have-been. They will require a healthy dose of luck in addition to flawless execution.

Peak Resources

By now I expect that most people have heard of the concept of peak oil. This is the observation that we have reached, or may soon reach, the historical maximum oil production. Like any prediction based on limited data, the timing is uncertain. What we do know is that the quantity of petroleum, in all of its naturally-occurring forms, is finite. Therefore there has to be a peak sometime.

Of course, peak oil can come about for reasons other than resource depletion. Examples include reduced demand (rampant environmentalism?) and an unexpected replacement energy source (fusion, high-efficiency solar). For the present discussion let's assume that the present demand trends continue so that we can focus on production. We know that oil -- or any natural resource, including gold -- cannot last forever. But how much is there, and how can we know?

These may be the wrong questions. Though studied extensively, the composition of the Earth's crust remains unknowable. It is simply too vast and sampling can only give us an imperfect statistical picture of its contents. It is in fact very likely that there is far more oil, natural gas, gold, and other valuable ores than is forecast in the most optimistic resource reserve calculations. That isn't the problem. The correct problem is one of economics: what it costs to access and extract the resources.

The marginal cost of extracting the resource is inexorably climbing. This is no surprise since profit-seeking companies seek the lowest-cost resources; this is the so-called low-hanging fruit. Resource deposits may be long ignored until the market price rises above the deposit's extraction cost, since this is the only way to achieve profitable production. There are abandoned gold mines in northern Ontario that are now getting a second look since their residual deposits may now be profitably mined because of current high gold price. A key measure to estimate the cost of mining mineral resources is the desired resource's concentration in the ore. As the concentration gets lower, more rock must be processed to produce a given output, and that increases the cost. As technology improves or the market price increases, lower concentrations become profitable. Get the price high enough and it is conceivable that even the gold in ocean water and river effluent could be profitably extracted.

Oil and natural gas have similar economics. With better and more-expensive technology, more impure and less accessible deposits become profitable. The northern Alberta oil sands (bitumen) is so impure that it is mined rather than pumped out of the ground. In addition to this expense, the heavy oil has to be upgraded before becoming suitable for refining into sellable products. This has a high price. The oil sands are only profitable with the market price of a barrel of crude oil that is somewhere north of $60.

Another deposit that is a common object of attention in the idiotic chain emails that pour into my inbox is the Bakken Formation, which straddles the US-Canada border. The theme here is the vast forecast reserves of oil and gas in this deposit, and how it can sustain the US economy on its own for a hundred years or more. While this may be true, it is irrelevant since the bulk of the deposits, perhaps 99% of it, is currently inaccessible, and even it were accessible the extraction costs would be far above current prices. Get the price high enough and it might as well not exist. The more-easily accessed parts of the deposit are now being extracted and, while significant, it is not going to solve the any country's supply needs.

As finite natural resources are depleted, their prices will rise. Not because of peak oil, peak gold or other trendy buzz words, but simply because extraction costs will continue to climb as we deplete the most accessible deposits. There is lots of any resource, if you are prepared to pay the price.

Even if we do not voluntarily reduce our consumption of these resources, the higher prices give innovators and entrepreneurs the incentive to develop alternatives. This is what is driving the solar industry today. It is still only marginally viable, but as oil and gas prices trend higher over the long term, investment into these alternatives will drive down costs through production volume and a steady stream of scientific discovery and technological innovation.

Better than any regulations, publicity stunts (like turning out the lights for an hour on Earth Day), or pleas for ethical business practices, it is basic economics that will ultimately halt exploitation of non-renewable resources. If you're an investor like me, there are lots of ways to profit from this since the prices of many commodities will continue to trend higher and emerging technologies (though not necessarily any particular company) will see their own prospects rise alongside resource prices.

Mobile Phone Wars: iPhone vs. Android

In the media's ever-present search for horse races -- the often meaningless comparison of numbers that signify little -- we have the comparisons between Google's Android-powered mobile phones versus Apple's iPhone. As usual, they tend towards trying to paint the burgeoning competition in the smart phone space as a winner-take-all war. This is nonsense, but that doesn't stop the headlines or the (deliberate?) misunderstanding by journalists. The latest figures published by Flurry make a good example.

The figure shown here is from the referenced Flurry publication that compares early sales of iPhone, Motorola's Android-powered Droid and Google own Nexus One (manufactured by HTC). In the largely superficial treatment given to Flurry's figures in the media you might think that Android is an iPhone killer. But again, that's only if you believe there can only be one survivor. Besides, the numbers must not be taken at face value; as we will see, a bald comparison of the numbers do not properly describe what happened, even if one does accept that there is a war going on.

First, we need to note (and here we are exclusively looking at the US case) that iPhone is exclusively offered via AT&T while Droid is exclusively offered via Verizon. Google sells Nexus One direct-to-consumer, although it so far only works on T-Mobile's network. These carriers are not equivalent in the size of their businesses and in their business practices, and Apple, Motorola and Google also differ in their market strategies.

Consider subscriber counts: the Q4 2009 figures for AT&T and Verizon are 85.1 and 91.2 million, respectively. These numbers are close so you might figure that the sales figures should be comparable (1 million iPhones and 1.05 million Droids in the 74 days following launch) since they have similar addressable markets (each carrier has about the same number of possible customers for these devices). However, this is a poor comparison because the iPhone launched in 2007. At that time AT&T had 63.7 million subscribers, which is a far lower number. While much of the media overlooked this important data, Flurry's own analysis (from which the above figure was taken) is both balanced and accurate.

2. Relative Subscriber Bases: Droid launched on Verizon, a larger network with more subscribers than AT&T, especially when considering AT&T's 2007 size (63.7 million at the time of iPhone launch) versus Verizon's 2009 size (89 million at the end of Q3). Additionally, there was pent up demand among the Verizon subscriber base for an iPhone killer, which is exactly how Verizon positioned the Droid. Finally, Verizon backed the launch with advertising support of at least $100 million.

To put the sales figures in their proper light, we need to compare penetration into their respective addressable markets. Done this way, we see something quite different:

• AT&T and iPhone: 1.57% (1 million phones for a subscriber base of 63.7M)
• Verizon and Droid: 1.18% (1.05 million phones for a subscriber base of 89M)

Now we see a significant difference, where the iPhone can be said to have outsold Droid during their respective launches. Of course even this calculation has its drawbacks since we have to consider the impact of marketing by Apple/AT&T and Motorola/Verizon before and after launch, and that the iPhone was something quite novel in 2007 while the Droid was heavily marketed as an iPhone killer to a new smart phone-aware population.

Look at the numbers however you like if your objective is to pick a winner, but in too many respects we are comparing apples and oranges. Better, I believe, to recognize that both products (really, iPhone versus the whole line-up of Android-powered phones) are both successful products.

This brings us to Google's own branded phone the Nexus One. Despite some of the hype -- although it generally isn't coming from the traditionally marketing-averse Google -- it is little more than the newest generation of Android software and mobile device hardware. The software will soon be available on all Android phones and the chips are beginning to show up on devices running other platforms. Nexus One is not that special since it will soon be matched and surpassed by other devices, as is typical in the game of technology leap-frog with which we are all familiar.

Whether the Nexus One is a flop or not (135,000 phones in the comparable period) really depends on how one chooses to interpret Google's (non-)marketing strategy, but a strong case can be made that it was not their intention to try to compete in the mass market against those other devices.

Now that Nexus One is available in Canada, we can all do our own comparisons. I have a Nexus One on the way and I am looking forward to get some hands-on experience with this latest salvo in the mobile phone wars. If nothing else, I'll have something novel to pull out of my pocket when colleagues trot out their iPhones and Blackberrys at meetings. It'll be interesting to see their reactions.

Cisco and the CRS-3 Hype

If you follow the technology business, you would have noticed it when Cisco announced that they would be making saying something titanic the next day that would change the internet forever. When I saw that, my initial reaction was to re-read it a bit more slowly and then I thought: huh?

Don't get me wrong. I have immense respect for Cisco. There was even a period of some years where if someone (like an investor) understood that your company was planning to compete against Cisco, he would immediately bolt. They were scary good, and the respect and fear they instilled in others was well-earned.

Cisco is still a great company, but over the years since the dot-com boom and bust they have become quite a bit more uninteresting. They still produce great technology and do very well at pleasing their customer, but they are no longer quite so scary. Competing against Cisco is now perfectly acceptable and can be lucrative if you have the drive and business scale to take a serious run at them in any of the market segments in which they operate. Of course they continue to dominate in the core router business, and as the internet and all things IP has grown so have they. In short, their success has been earned.

The problem is that they have always found it challenging to break into new markets that are higher in the value chain, up beyond the basic routing of bits. They bought their way into many markets and muscled into others with brute force. Doing this has, for example, put them inside consumers' homes and made them a strong player in the enterprise and carrier VoIP market. All this highlights just how different the IP infrastructure market differs from the service and application markets. It's hard to be great at everything.

Maybe that explains why they took the hype route in announcing the CRS-3. It looks like a great product with excellent engineering, but revolutionary it is not. It is important to carriers since it allows them to evolve their existing base of Cisco routers without the higher costs (capital, operating, and manpower) of running more CRS-1 systems or switching to another vendor. I think Lightreading nailed it pretty well:

"...the CRS-3 is a CRS-1 outfitted for 100-Gbit/s interfaces...The 40-Gbit/s limit was a function of the switching fabric available on the CRS-1..."

It's an incremental improvement that I'm sure their customer have been demanding for some time. It is not a product that impacts carrier customer -- like you and me -- in the slightest. It isn't even that the capacity of the internet would choke with all those video streams to mobile smart phones, since there are alternatives that are perhaps more expensive or less convenient for the carriers. It am also quite certain that its announcement was not a surprise to any of the targetted carriers since you can be sure they've been promoting it as vapourware for a long time, if only to dissuade the carriers from switching to Juniper routers.

So it's great that they can now say "me too" when it comes to 100G Ethernet, and it's even better that they can do it by popping in faster switch cards and interfaces in the shelves of existing frames. They can even reduce the power budget, which is an increasingly burdensome expense in data centres.

Their claim that the CRS-3 cost them $1.5B in R & D is almost certainly bunk. They probably threw everything except the kitchen sink under that umbrella to add to the hype. Cisco does spend that and much more on R & D, but it is an investment that applies to many more products, both current and future.

Cisco remains a great company, especially a router company, and this is unlikely to change. They are very good at building the plumbing of the internet but, unfortunately for them, most people only notice the fixtures in their homes, not the plumbing that makes them work. Cisco's name isn't found on many fixtures.