Sunday, July 25, 2010

The Psychology of Marginal Cost

One of the side-effects of moving internationally is that one is typically required to completely re-evaluate the set of services which we are generally accustomed to - e.g water, power, telecommunications, transport etc. Partly because traditional utilities are very local and partly because the circumstances of life are more or less subtly organized differently in different places. One such difference can be the available pricing models for a particular service - most commonly some form of flat-rate or metered pricing.

In areas where both flat-rate and metered pricing plans exist, analysis often shows that even though many consumer prefer flat-rate, the typical user would be better off with metered pricing, as only very few heavy users manage to fully use or "abuse" the plan.

Consumers often quote predictable cost and "no bad surprise" at the end of the month as a key benefit of flat-rate pricing. But another interesting observation to take into account is that in most cases, flat rate pricing stimulates increased usage. The real motivation for consumer to choose flat-rate pricing, specially for things related to fun and entertainment, might also be to get the unpleasant financial considerations out of the picture once and for all and not to remain as a kill-joy, nagging question each time the user feels like making use of the particular service.

For us, the most significant area of changed behavior seems to have been transportation. In Switzerland there is a flat-rate pricing option for all public transport, popularly referred to as the GA (in German). For about 200CHF per month and person, this allows to hop on any train, bus or boat anytime, anywhere in Switzerland. On the other hand, we don't currently own a car. This decision was made easier by the existence of Mobility CarSharing - a dense and well established car sharing service. The signature red Mobility cars can be found at almost any train station and there are about 6 cars available in our neighborhood, just a few hundred meters from our front-door. Pricing is a mix of hourly rent and per-km charge, which clearly encourages a networked usage of trains for long-distance, mobility car for "last mile" service. Pricing is around 3CHF/h (0.6CHF during night-time) and 0.5-1CHF per km charge depending on the car model. This is clearly not cheap and for a long week-end trip breaks about even with a normal rental car.

However, the membership based system and the dense network of dispersed self-service locations offers hugely better pickup and drop-off experience than any rental-car company possibly could. Having a small child, we knew that the optimized combination of train and car would likely not work, due to having to lug around a heavy ECE R44 group II compliant car seat in addition to all the other stuff, small children generally come with. Yet assuming the cost of owning our own car to be at least about 1000 CHF per month, we could literally take a car for each week-end and still come out ahead.

After almost a year, it turns out that we have used the car less than half a dozen times (mostly going to IKEA or other furniture moving activities) and did all other travel including many spontaneous excursion by public transportation. One key factor is that the decision to take the care will most likely result in a bill in the order of hundred(s) of CHF, while the marginal cost of using any public transportation for any time or distance is for us now zero CHF. Even with the streamlined procedures, reserving, picking up and dropping of a mobility care requires some level of planing, preparation and discipline, while the combination of GA and Google mobile transit directions provides a near frictionless level of spontaneous mobility (at least between town centers). Yet, despite the very dense Swiss transportation network, a typical trip still takes us much longer by public transport than it would by car - but maybe because I don't particularly like driving, I am more willing to put up with time lost having to wait for a connection.

Despite a bit of traveling in the last year, we still did not reach or exceed the cost of the GA compared to the optimal strategy using individual fares. Yet we would still consider it as a success, by putting a lot of emphasis on convenience (not having to figure out what the idea fare is, and how each ticket vending machine works...) as well as not having any excuse to avoid going out and discovering our new surroundings - encouraged by the psychology of a zero marginal cost for each trip.

Friday, July 23, 2010

Android 2.2 - Froyo

I just finally got the new Android 2.2 release for my Nexus One (it's a long story...). Most of the significant features of this release are behind the scene like increasing java execution performance through just-in-time compilation or increasing javascript performance in the browser by using the V8 JavaScript engine from Chrome. Not sure I really notice much of a difference in everyday use, since I mostly use apps which are UI and framework bound for their performance (no CPU heavy games...) and most web-pages are light in JavaScript and browser performance is limited by network and rendering performance. But still these are very welcome optimization to help improve the platform overall.

There are a few small enhancement - most significant for me is the ability to switch auto-correct/complete languages for the on-screen keyboard on the fly, since I write emails and SMS in multiple languages on any given day. There is now also finally a switch to disable the use of cellular data independent of any other function, which had been one of my complaints for a long time.

New on the platform side are a few services like the ability to back up application data in the cloud, make use of the SD-card for installing a lot more apps than what the capacity of the internal flash memory can handle or the C2DM notification service discussed here earlier. Any of these require applications written with the new SDK and using these new features to show their full potential.

But I am most excited about the official support for tethering in Android 2.2 - i.e. the ability to use the phone's cellular connection as an uplink for other devices either through USB or wifi. While USB would clearly be better from a battery life perspective, the number of devices which support networking over USB via the phone is a lot more limited than the number of devices which support wifi: today pretty much any internet capable device seems to support wifi. When turning on the mobile wifi hot-spot feature, the phone is basically acting like a wifi base-station, creating a wifi subnet which other devices can join and access the Internet by sharing the phone's cellular data connection.

A quick test with speedtest.net from my tethered laptop vs. the speedtest Android app running on the phone shows no noticeable throughput degradation by tethering (about 2.7Mbps upstream and 1.5Mbps downstream on Orange CH 3G service from our house in Zürich around midnight).

Friday, July 2, 2010

Push Notifications for Android

After struggling with a few apps which use lots of battery and network resources while trying to sync half the Internet onto the device, one wonders if Apple didn't accidentally have a point with their claim that most apps don't really need background processing as long as there is a way to push background notifications to the device.

This leads to a split application design, where part of the application resides on a server in the Internet, doing whatever the background service on the device would be doing, but with a lot less worries about power and bandwidth. If there is something new and interesting, a small notification is pushed to the device to alert the user, that there is something worth looking at. As long as the device has network connectivity when the user acts on this notification, the details of this notification might be about is loaded on demand. If mobile networks are ubiquitous and fast enough, the resulting experience is almost as good as an app which continuously loads and caches content in the background - and a lot friendlier on battery and network usage.

The upcoming Android 2.2 release includes support for the new Android Cloud to Device Messaging Framework, which promises to do that and then some in a generic fashion.

If an app on an Android device wants to receive some particular notifications, the application registers to receive the notification events from the Android OS and requests an authorization key from the C2DM framework and sends it to its backend server in the Internet (e.g. by http request, SMS or whatever). When the server wants to push an event to such a registered device, it uses the authorization key to contact the C2DM service, which will then push the notification to the device.

One of the sample apps included with the framework - Chrome to Phone - shows how this could work. From a Chrome browser extension, the user can choose to send the current page to their Android device, where the page then can be loaded from an entry in the notification bar:



But since Android does support background processing, this framework isn't limited to simply pass on notifications to the user. An app could still choose to act on some incoming notifications in the background, i.e. by fetching and caching the content which the notification event refers to. The frameworks seem flexible enough to create all kinds of new applications which create a more seamless integration of mobile and web-based applications.

Friday, May 21, 2010

Thoughts on "Making Sense of Privacy and Publicity"

This year's SXSW keynote by danah boyd is probably one of the most insightful contributions to the debate on privacy and social networking. For those who have not yet seen it, the rough transcript can be found here. It puts a finger on so many important points, that it should be required reading for anybody who wants to work on consumer web services.

To summarize a key point: in real life things are usually not as simple as they seem. And that's bad news for the technocrats who typically build and run the virtual environments where social interactions are taking place online. Engineers and scientists like to simplify and standardize problems, apply Occam's razor, optimize systems along the dimensions of an assumed known quantitative model etc. The operators of today's large web properties study and analyze their users behavior and think to understand them better than the users understand themselves, but behind the user behavior observable from web logs are layers of significance and meaning which are completely hidden from this behavioral analysis.

In every society, people daily strike many a delicate balance between engagement and guardedness. We learn social rituals, what is appropriate in a given situation and what kind properties to expect from a certain context or environment. The boundaries of what is private and what is public are blurred. Some of the most private, intimate and confidential discussion happen in very public places - on park-benches on a warm summer night or in Washington D.C parking garages. In the offline world, experience allows us to judge how an environment will support our interactions - and in many jurisdictions, altering those properties through hidden means like microphones, telephoto lenses etc. is explicitly illegal as a violation of privacy rights.

In the online world things are a bit more tricky. We don't usually know as well how these virtual environments really behave and their properties are really easy to change by the people who build and run them. To make things worse, the Internet (almost) never forgets. While an intimate discussion years ago on park-bench has faded from the observable universe long ago, the same discussion done by IM may remain visible somewhere in a chat log forever. Users often assume rightfully that their daily lives are mundane enough that nobody will bother to explicitly track them through the digital noise. Safety by numbers and hiding in plain sight often works surprisingly well. Just because something is not explicitly block from access does not mean that the creator necessarily wants everybody to see it.

Assuming that users somehow find a delicate balance on how to operate within the virtual spaces formed by social networking and other web 2.0 richly interactive services, some of the worst things which the operators of these services can do to hurt their users is to change the rules on them. Since we do not know what kind of balance each user has found to make things work for themselves, it is hard to predict how any change might affect them. And since there is a lot of data in the system, changing the rules can even be retroactive: a spotlight suddenly shining into a corner of the virtual world where it was not supposed to, according to the expectations of the user.

Unfortunately the current best practice of software development is based on embracing change. The software as a service model allows to release early and release often, since nobody knows what will really resonate with users. In the end, we get services which are more sophisticated and more integrated with our daily lives than ever before, but at the cost of the eternal beta.

In order not to violate users sense of privacy, any changes which shift the fabric of our virtual online worlds that might affect the visibility or exposure of anything must be considered very carefully for unintended consequences.

As a fundamental principle, there should be no "ex post facto" or retroactive change to the visibility and exposure of anything without the most explicit and informed consent of the user. But even when operating diligently by such a high standard, accidents are bound to happen from misjudging the impact some changes may have in the user's world.

Ultimately as the Internet becomes more social, we need to better understand the social dynamics, conventions and rituals of its usage. But unfortunately that's not something that the introverted computer geeks who have so far built the foundations of the social web are particularly good at.

Thursday, May 20, 2010

From UGC to UCC

I have noticed, that a good part of the articles I read online have been suggested by members of my various social networks. Maybe it is a part of the true utility of social networks to be a platform for "User Curated Content".

While the web in its first phase tried to mirror the offline world, by moving every brick and mortar institution and service online, the so called web 2.0 promised a new world of participatory media, where everybody can create content. While digital media have drastically lowered the production costs, the web has driven distribution costs to near zero. Looking around on blogging sites, flickr, YouTube or other cornerstones of the "User Generated Content" revolution there are some seriously talented people out there! Some people have managed to make a mark, some even managed to make a living or become minor Internet celebrities in some field. Some other stuff is whimsical, funny or personal. There are unexpected viral hits or observers who happen to be at the right place at the right time and turn into citizen journalists. But the vast majority is just plain boring and inconsequential rubbish.

More so than ever, the problem has become discovery - i.e. finding the stuff that's relevant, interesting, worthwhile, stimulating, satisfying etc. at this point. When production and distribution of content is expensive as in traditional media, there are plenty of people whose primary role it is to make choices of what is being produced and distributed on behalf of their audience. They are called curators, editors, DJs, program directors, executive producers etc. and they are often the most most well know, prestigious (and feared/hated) people in their organization.

Curators make choices which works of art are on display and which ones are in storage, the ones at leading institutions even define what is considered art, based on what they acquire for their collections. Editors in chief decide what stories are being printed and define our perception of what is news. In a situation of scarce resources the difference between a curator and a censor are often only the nature of their intentions (educate and enlighten vs. oppress). On the Internet, the role of a curator is different. There is (near) infinite wall-space and everything which exists can be exposed - but because of that often not be seen or found by anybody in the sheer mass of stuff out there.

There are a few successful strategies for finding something in this giant heap of digital noise. Contextual search ranking revolutionized web-search in the late 90ies by creating algorithmic determinations of what is presumably more interesting or relevant to a particular question. Clustering algorithms can help find similar things to something we like and recommendation engines can suggest things which people like me have liked.

However, in a world dominated by the chatter of millions of undistinguished sources, search can often fail to find the nuggets in the trash heap. And recommendation engines only reinforce my current point of view. How can I learn, grow, be surprised and intrigued if I am only ever fed things recommended by people like me? Why should my taste and judgment be any good... I would rather get recommendations from people who are smarter, more knowledgeable, more stylish and more plugged in to a particular filed, but that's hard to decide by algorithmic means.

So we are back to curators. Or editors, guides, teachers, gurus, opinion-makers, trend-leaders, talent-scouts or whatever we want to call them. Relevance is no longer defined globally but based on people whose judgment we trust and respect. Social networks can be a source of such relevance, following the age-old patterns of world of mouth among friends and family. Or more powerful would be asymmetric social networks where we can find somebody whose judgment we respect and "follow them", without them necessarily having to know us.

In fact many blogger are in fact more editors or curators than creators of original content. This is even more so with micro-blogging systems like Twitter, which many consider the quintessential asymmetric social network. And Wikipedia is probably the most high profile project, where domain experts can live out their inner librarian and do so with great determination.

Given the importance of curators, I think there is still too much emphasis on content creation. The real challenge today is to mine the piles of digital trash for nuggets of gold which most certainly exist in numbers never seen before. We already have more content than we ever know what to do with, but there are not enough platforms and frameworks where people who would like to organize it could shine and be recognized. Part of the problem is that when it comes to derived works, copyright gets really murky and it's hard to say who should get credit (or even paid) for what. But it is time for online curators to get more respect and for librarians to step into the limelight. And this could easily be one of the next big things for online digital media platforms.

Tuesday, May 18, 2010

A day in the life of the Internet

Todays top suggestions on google.com search for "How do I" are:
  1. how do i delete my facebook account
  2. how do i find my ip address
  3. how do i get a passport
  4. how do i know if im pregnant
  5. how do i love thee
  6. how do i look
Out of which only #5 has a relatively straightforward answer:
... Let me count the ways.
I love thee to the depth and breadth and height
My soul can reach, when feeling out of sight
For the ends of Being and ideal Grace.
I love thee to the level of everyday's
Most quiet need, by sun and candle-light.
I love thee freely, as men strive for Right;
I love thee purely, as they turn from Praise.
I love thee with a passion put to use
In my old griefs, and with my childhood's faith.
I love thee with a love I seemed to lose
With my lost saints, --- I love thee with the breath,
Smiles, tears, of all my life! --- and, if God choose,
I shall but love thee better after death.

Sonnet 43, Elisabeth Barret Browning

Friday, May 7, 2010

IT != IT - the Case for a Differentiated Immigration Policy

The Swiss government recently reduced the quota for work-permits for applicants from so called 3rd-states - which typically means countries outside the EU and not covered by the free-trade treaties between the EU and Switzerland. After a highly publicized protest led by high-tech companies like Google, Microsoft and IBM, the Swiss government has rather quickly reverted its decision.

In the midst of a recession with higher than usual unemployment and increased levels of immigration from the EU following the free-trade agreements, the general mood in the population is not very supportive of any increase in immigration quotas. This is seen as yet another attempt by greedy corporation to undercut the Swiss standard of living by importing cheap labor from overseas - typically from south-east Asia in what is generally by called the IT or information technology sector. How can there be a shortage of IT labor, if almost everybody knows someone who is unemployed and supposedly somehow "in IT"?

As with most controversies, there may be some truth to the matter of companies trying to use immigration to depress labor costs, but the crucial core of the problem here is the ability for for companies who operate global R&D facilities in Switzerland (like the companies named above) to be able to attract the best possible talent in a particular field - regardless of skin-color or country of origin.

Since information technology has permeated about any aspect of not just business, but also increasingly personal live, the so called IT sector has become large and diverse that saying somebody is "in IT" is about as meaningful and descriptive as saying that somebody is working in an office.

The vast majority of IT jobs are about supporting and customizing systems and applications based on the specific needs of their users. The most visible IT workers are PC technicians or system administrators, which almost everybody knows first hand from their daily work. Or armies application developers which work on big in-house IT projects for large corporations like banks or insurance companies - either as employees or as contractors from large IT services companies like Accenture, IBM or Infosys on behalf of local clients. Most of what is going on here, is indeed not rocket science from a technical point of view. The key stake-holders are not technology companies, nor are they interested in technology per-se. They rather consider it a necessary evil, a cost center which they would like minimize. No matter how high-tech certain IT service providers are giving themselves for the public image, in reality they are trying to be as technically unspectacular and conventional as they can be in order to minimize the risks of implementing something which has basically been done before many times over in slightly different forms. As no two organizations are exactly the same, the IT systems used to support them are also slightly different which causes all this effort and duplication. For many of these jobs, the ability of communicating with the users of the system and understanding their application domain is a lot more important than raw technical skills and knowledge. It is rightfully debatable to what degree immigration vs. increased education and training should help resolve the general shortage in this still fast growing sector.

But there is also a very small segment of the IT industry, which is the true high-tech sector. This is where the technological innovation happens. These are the companies and people, why build the core pieces like operating systems, database engines, computer chips, programming frameworks or communication equipment. This is typically also where the value-add is most concentrated and companies like HP, Apple, Cisco, Intel, Microsoft, Oracle or Google have famously propelled their founders and investors into the top league of the worlds most wealthy people. This is where the coolest and sexist jobs are for the technically inclined, ambitious, talented and well educated in the IT workforce and the companies who are being seen as the avantguarde of technological innovation can typically get a pick of whom they want to hire globally.

This kind of jobs are also typically concentrated in a few select places around the world, most prominently in Silicon Valley, because this is where the necessary key talent can be found. However this is not because the people born in Santa Clara County are somehow smarter than the rest of the world, but because of migration. Silicon Valley has a share of foreign-born population way above the US average at about 40% (10% US average) an a rate of over 60% among engineers and scientists in the Valley. And this hides an equally significant domestic migration within the 300 million US population, where many with advanced degrees in engineering and science have moved to Silicon Valley from all over the country, if they want to play in the top league of their field.

Even though they pale in scale and importance next to Silicon Valley, there are a number of secondary clusters of high tech excellence around the world. Switzerland is reasonably well positioned with a strong tradition of industrial innovation going back to the industrial revolution in the 19th century, two technical universities who often appear among the highest non-anglosaxon institutions on many league-tables and a number of high-profile R&D labs. Some of them by domestic champions (pharma & machine industry) and some number of US high-tech companies (e.g. IBM, Google, Microsoft, Cisco) who have decided that Switzerland is a good place to hire some of the top talent, who for some reason doesn't want to move to Silicon Valley... It would be hubris for a country of barely 7 million to assume that a significant part of the worlds leading experts in any particular field could be produced domestically, no matter how strong a culture of excellence or how strong the confidence in the local education system.

Any organization which is at the same time highly specialized and world class must necessarily be able to recruit globally from the best talent possible in that particular field, whether it is a top-ranked symphony orchestra, a premier-league football club, an elite-university or a world-class industrial R&D lab.

The reasons why global companies have chosen Switzerland as a place for global R&D is only partly because of a strong technical tradition, good universities and some number of key talent already there, but also primarily because of the highly rated quality of life, including reliable public services, picturesque landscape, low personal income taxes, safety, stability and a generally pragmatic government. Basically a place where it is relatively easy to convince people to relocate to, who otherwise have plenty of choices, options and other offers.

In the grand scheme of things, these few world-class labs will only employ a few 100 to few 1000 highly educated specialists and as such have very little impact on the overall employment or immigration situation. But they generally contribute a disproportionate amount to the economic development through prestige or intensified interactions and networking with other local firms and universities. For the one key ingredient - the ability to attract top talent, they need to to be able to recruit internationally with minimal restrictions and interference. At this point it is up to the Swiss government to either leverage its current reasonably strong position in the competitive global knowledge economy or to snatch defeat from the jaws of victory. E.g. through misguided pandering to the populist right-wing on immigration and to the left in the form of misguided attempts at labor market protectionism.

Organizations who can credibly make the point that they are in that global league, recruiting for the top talent in their field, should be except from any restrictions and quotas. In addition there should generally be a priority visa category based on education, skills and experience compared to the best in their filed - similar to the "alient with extraordinary ability" visa in the US. To go a step further - anybody who graduates with distinction from any university in whatever to-N global league-table, should have the automatic pre-approved right to a work and residence permit should they choose to come. (To calm the shrieking voices of panic on the right: very few would actually come, since they typically have plenty of opportunities and lots of other good offers. )