Thursday, July 23, 2009

Collecting Android Logs

One of the problems debugging Android apps in the field is that the system error messages are quite generic and meaningless - e.g. Application not responding or application has stopped unexpectedly. All the interesting details like the java exception stack trace in the case of crashing apps are logged to the system log. This log can be looked at by running the "logcat" command from the background debug shell, usually by connecting over USB via the "adb" command from the Android SDK.

But what to do if a problem occurs in the field, when you don't have a computer handy with USB cable and Android SDK installed? I recently came across a few apps on the market which address this issue by running logcat in the background from the application, capture the output and bringing up an email message with the output to be sent to somebody - either yourself or the developer of the crashing app, who had been desperately asked to see the logs.

The source of one of the Log Collector apps is available online, and I was obviously curious how this is done. I had always wanted to add a log viewer to NetMeter, since seeing the logs is very essential to trouble-shooting anything going on with an Android device. Looking at the source-code for logcat, it seemed challenging to re-implement something similar in java as an Android application.

What I didn't know, is that it seem to be possible to run any native unix commands from within and Android Java application - even on regular production devices, assuming the application task has the necessary permissions to execute the particular command. Simply running the logcat command seems indeed to be the easiest way to get the system log from within an application. If I can find some time, maybe I'll add the logviewer to NetMeter after all.

Friday, July 17, 2009

Virtual Phone

Similar to the virtual postal address setup, we are looking to keep a virtual US phone number while being overseas. Reasons for this are to give our friends and family in the US a permanent local number to call, no matter where are at a particular point in time. Having a US number for outbound calls is can also be useful to call US based 1-800 numbers - e.g. to call a credit card company and yell at them to stop sending spam to our vitual mailbox for example...

The simplest way to do that would be to get VoIP phone service from a provider like Vonage and the simply pack up the VoIP adapter and US phone and install it wherever we are - using the proper power converters.

However, we are doing something slightly more convoluted. Even though Skype became famous for offering free computer to computer calling using a peer-peer, the now also offer gateways to the plain-old telephone service at a fraction of the cost of most other VoIP providers.

For $9 per month, we have unlimited calling to the US and Canada plus two phone numbers, one in the US and one Switzerland. We can also set up forwarding to any number (free in the US, about 2c/min. anywhere else with the current subscriptions) so that any Skype calls, wheter from a computer user or from a call to either of these numbers will be forwarded to our designated home phone if we are not running the Skype anywhere on a computer.

Linking checkins to tickets (SDI 07 VII)

There are two crucial pieces information related to every change to the project's codebase: the what and the why.

Thanks to using svn for version control, all the details regarding what a change is are being automatically recorded in the form of an atomic changeset which moves the codebase from one (hopefully) consistent state to another.

Recording the why requires a bit more work. In this simple workflow, we are using Track tickets to document and track every single legitimate reason for making a change to the codebase - whether they are bugfixes or work items related to new features or enhancements. If every legitimate reason to change the code is represented by a ticket, then for each svn changeset there should be one or more tickets which shows why this change was made. Code changes shoudl be done in small incremental steps, but the reason for these changes may be a fairly large an complicated project requiring many small code changes. Tickets can also be used as threads for resolving and documenting design decisions.

In the Trac web interface any reference to other Trac "objects" like svn changesets, tickets or wiki entries are automatically represented as hyperlinks. In order to link a svn changeset to its related tickets, all we need to do is to mention "#<tktnum>" in the commit message for this change. In order to automate the reverse linkage from the ticket to the changesets which refer to it, Trac provides an optional script which can be installed as a svn pre-commit hook to post a message to the ticket log with the changeset ID.

There are two scripts which can be downloaded from or whatever version of Trac is being used. The first one is trac-pre-commit-hook which can be used to automatically enforce the policy that each svn changeset message must contain a reference to an open Track ticket number in the form of expressions like ”closes #5”, ”fixes #34”, ”addresses #456” or ”references #4”.

The second one is trace-post-commit-hook, which will post messages to the corresponding tickets referenced this way in the commit message after the commit has been successful and potentially changes the state of the ticket in the case of "closes" or "fixes" references.

In order to install one or both of these hooks we need to create the following /data/sdi07/svn/hooks/pre-commit script:

LOG=‘/usr/bin/svnlook log -t "$TXN" "$REPOS"‘
/usr/bin/python /data/sdi07/svn/hooks/trac-pre-commit-hook \
"$TRAC_ENV" "$LOG" || exit 1
exit 0
and add the following to the already existing /data/sdi07/svn/hooks/post-commit script:
/usr/bin/python /data/sdi07/svn/hooks/trac-post-commit-hook \
-p /data/sdi07/trac/ -r "$REV" -s ""

Tuesday, July 14, 2009

Virtual Mail

We are about to move to Europe for a few years and during that time, we need to be able to maintain a virtual presence here in the US, to make going back and forth more easy, to stay in touch with friends and family and to take advantage of the greatest consumer paradise on earth, where everything is available for a buck or two.

An important part of maintaining a presence is to have a US mailing address - e.g. this required to have a US credit card and to have stuff shipped to you. For certain things a PO box would do, for others you need a real street address - e.g. to accept packages.

Traditional mail forwarding places have existed for a long time, basically private mailbox operators who also provide the service to periodically mail the content of the mailbox to a forwarding address.

Shipping overseas is really expensive and the idea of getting a box of junk-mail sent to us every months or so does not seem too appealing. Also if something is really important, it might also be urgent and can't wait until the next delivery.

Fortunately there are also alternative which integrate paper mail more closely with e-mail. Most of the time, I don't need the physical piece of paper which is being sent in the envelope, just to know what is in it or to know as soon as possible that it arrived is often good enough.

There are a few services for expats and other nomadic users without a steady mailing address which offer online remote mail management. This typically involves scanning all incoming envelopes and sending email notifications when something new arrived. The user can the log into a web application - their virtual PO box and decide what to do with the content: ship it to another forwarding address, shred it and throw it away or in some cases, have it opened and the contents scanned and made available through the virtual PO box.

We ended up choosing Earth Class Mail because of their emphasis on scan & discard vs shipping. Yes, it is a bit scary to have random strangers open up your mail and it is a bit hard to be trusting a startup company in this current financial climate that they will not certainly disappear of do some very shady things out of desperation.

Fortunately all my financial accounts support online interfaces as well and most of them are coming around to let users opt out of any paper statements and transaction records - green is in right now! Unfortunately at least one of them insists in spamming me with silly offers, which I hope is something I can get them to stop so that it might be easier to recognize the envelope with the new credit card or the new PIN code and selectively forward it, without having to open them.

And for a lot of other things, my life just isn't interesting enough to provide the workers in the mail processing facility with a lot of entertainment and for the rest I hope they would be too busy to engage in large scale identity theft.

After setting up the account, we had to send in quite a bit of paperwork - certified copies of passports and other IDs for the US postal service to deliver mail to a third party mail handling agent. After all this had been received and processed, we now have both a street address and a PO box address set up to choose from. PO boxes tend to draw less spam mail, but street addresses are necessary for all those situations where a PO box address is not accepted.

I sent a test letter to both addresses and was quite surprised how quickly they envelope pictures appeared online. After I requested open & scan, the images of the content appeared online a day or so later. Despite being located in the Pacific northeast, they must have local processing on the east-coast as well, since the letter mailed to an east-coast address appeared online a day or so ahead of the one sent to the west-coast address - and only about two days after I had mailed in New York City. I was quite impressed with the speed.

We have not switched over any of our important mailings to this new virtual address(es), so I can't really say yet how well it works. I am a bit worried that once the address leaks out onto the commercial mailing lists, we will get so much spam mail to eat up our monthly envelope scanning quota just for that. I guess we'll just have to become more aggressive about fighting spam.

Sunday, July 12, 2009

Manhattanhenge II

Today was the second occurrence of Manhattanhenge this year - both equally before and after the summer solstice (June 21). I didn't take any pictures this time, largely because it was cloudy and rainy tonight.

Thursday, July 9, 2009

Component Architecture, Android Style

Android has a nifty component framework, where each screen - called "activities" should be self-contain can be called up by anybody through an event distribution mechanism called "intents". Activities can also register to handle arbitrary intents, which allows applications to delete certain functionality without even knowing which app is going to provide this functionality. Certain intents are pre-defined - e.g. the NoiseAlert app uses the ACTION_CALL system intent to delegate the making of a phone call to the dialer app typically, or whoever can handle it. Anybody can define new intents, but to pull off useful delegation of functionality between completely unrelated apps is usually a bit harder to pull off.

But here is an example of that. A while ago, a user of the BistroMath tip calculator was asking whether I could add the functionality of recording and tracking dining expenses over time. I can see this would be a very useful functionality. But I also think that mobile apps should follow the Unix philosophy, where each program does one thing well and interconnect with others to provide more complex functionality.

Basically I didn't want to build expense tracking into BistroMath, but wouldn't mind interfacing with a specialized application through the intent framework. So I was checking out expense tracking apps on the market (there are quite a few) and emailed the authors of a few which I liked and thought would work reasonably well for what the user had requested.

Within 24h the author of Funky Expenses had replied with a proposed intent interface and a new version of his app which implemented it. I just added a little code to BistroMath to trigger it and there we have a tip calculator with expense tracking capabilities.

For anybody who wants to support the same intent in their expense tracking application or to support calling it from tip calculators or other financial apps, here is an example of the caller interface:
Intent launchIntent = new Intent();
launchIntent.putExtra("com.funkyandroid.DATE", System.currentTimeMillis());
launchIntent.putExtra("com.funkyandroid.PAYEE", "Per Se");
launchIntent.putExtra("com.funkyandroid.CATEGORY", "dining");
launchIntent.putExtra("com.funkyandroid.AMOUNT", "1532.42");
try {
} catch (ActivityNotFoundException e) {
Toast.makeText(this, "No application found to handle expense reporting functionality", Toast.LENGTH_LONG).show();
Any of the extra attributes can be omitted and should then appear blank or as default values in the input mask of the event tracking application which is called up. The intent is also documented at the OpenIntents intent registry.

Wednesday, July 8, 2009


As the number of apps on the Android Market is growing, finding interesting apps is increasingly becoming harder. Developers have a very limited opportunity to promote their apps - a name, an icon and 325 characters are all there is to tell the user what the app is about. No screen-shots, no release notes, no FAQ, no back-channel to respond to user's comments. And worst of all, there is still no web interface which puts whatever little information there is on the market online and available to search engines.

However useful smart phones are supposed to be the speed, power and comfort of a full-size laptop or desktop computer are hard to beat when it comes to searching and browsing large amounts of data. The quality of the built-in search function in the Android market is a bit spotty to say the least - why not let people use real powertools like an Internet search engine to find android apps?

For a while there has been Cyrket to expose the content of the market online - presumably by reverse engineering the market's client-server protocol. Recently, Cyrket has gotten some competition in the form of AndroLib with a slicker, more polished appearance and some extras like the ability to upload screenshots.