Skip to main content

Posts

Showing posts from 2009

Android Log Viewer

One of my latest favorite discoveries in the Android market is aLogcat , a must-have for Android developers and power users, who want to know more about what is going on on the device. It is named after the logcat command which can be run in the debug shell, typically via the adb tool from the Android SDK, which requires the device to be connected to a host PC through a USB cable. aLogcat allows to display a log console on the device itself, color coded by levels with options to filter by levels or arbitrary substrings. By default the console updates continuously with new messages as they appear in the log, but it can also be frozen to allow scrolling back through the log history without interfering screen updates. Since logs can also be sent via email, it subsumes the functionality of earlier log collector apps. Now that the number of devices, configurations and version of Android are exploding, it is less and less likely that a developer can reproduce a particular problem, since t

Online Backup

Since we recently moved, my current backup system has become some what undone. I have not been able yet, to reactive my linux home-server , since neither its power input nor its TV signal output works in the new environment. But since backing up to an aging piece of low-cost hardware running an obsolete version of an OS, which I also happen to use for experimentation does not leave the kind of warm fuzzy feelings which one typically expects from a backup solution, maybe it was time again to look around for another solution. From a maintainable and reliability point of view, it would be better to store the backups in the cloud, rather than on a single computer in the same room. On the other hand, sending the data out of the room opens up some serious privacy concerns The solutions for online backups on mac are still a bit limited. There are some portable solutions using Amazon's S3 cloud storage service or the very open-protocol based rsync.net service, which could have supporte

Customized Call Routing

I now have a cellphone plan, where all outgoing calls are metered and very expensive, except for unlimited calls to 3 favorite numbers, which are included in the plan. For our international calling, we are already using a discount carrier activated through a dial prefix, which also supports a local land-line dial-in access to be usable from mobile phones. By declaring the access number as one of my favorites, I can make unlimited calls from my mobile phone at the substantially lower rate of the discount carrier. The only problems is that making calls through the indirection of a voice-prompt menu is very cumbersome! This is where Phonecard Express comes in. This highly customizeable application inserts itself almost transparently as a filter between the Android system dialer and the telephony subsystem. Whether a call is made from the dialer, the address book or any other intent, the call setup is intercepted and potentially routed through a calling card service. In addition to suppo

Virtual Phone, part II

I started using the new Google Voice service as my standard US phone number. Since they currently only support international calling (billed rate), but not international forwarding, the current setup is a bit suboptimal and roundabout: from Google Voice, calls are forwarded to my US based Skype online number, with my skype account itself being forwarded to my current cellphone. The only catch is that the post-dial-delay of this whole contraption is too long for me to pick up the call before it goes to voicemail and since the voicemail delay is not configurable there isn't much I can do. I once managed to catch a call today, by leaping at the answer button on the first ring - it was a wrong number... At least I get an email notification right away, when somebody leaves a message. I could probably just as well turn off forwarding and just use it as a email based voicemail-box.

Monitoring Network Usage

Some people seem to be using NetMeter for continuously monitoring their network usage against a limited volume data-plan. Since we are now moving to Europe, where stingy data plans with only a few 100Mbs included per month are the norm, I now also have a sudden interest in measuring my monthly data usage. Since NetMeter is a trouble-shooting tool, oriented towards very detailed tracking of network usage over the last few minutes to see what is going on right now, it is not really very suitable as long-term bandwidh counter. The ideal network usage counter should be low overhead and run continuously in the background - e.g. start itself at boot time. In its most basic form, it should show the current monthly total of cellular data usage. It would also be nice to enter the specs of the data-plan to track agains: how much is included per month, what is the rate above that limit and when does the monthly limit reset. For the current month, the ideal application should always show how curr

Collecting Android Logs

One of the problems debugging Android apps in the field is that the system error messages are quite generic and meaningless - e.g. Application not responding or application has stopped unexpectedly. All the interesting details like the java exception stack trace in the case of crashing apps are logged to the system log. This log can be looked at by running the "logcat" command from the background debug shell, usually by connecting over USB via the "adb" command from the Android SDK. But what to do if a problem occurs in the field, when you don't have a computer handy with USB cable and Android SDK installed? I recently came across a few apps on the market which address this issue by running logcat in the background from the application, capture the output and bringing up an email message with the output to be sent to somebody - either yourself or the developer of the crashing app, who had been desperately asked to see the logs. The source of one of the Log Col

Virtual Phone

Similar to the virtual postal address setup , we are looking to keep a virtual US phone number while being overseas. Reasons for this are to give our friends and family in the US a permanent local number to call, no matter where are at a particular point in time. Having a US number for outbound calls is can also be useful to call US based 1-800 numbers - e.g. to call a credit card company and yell at them to stop sending spam to our vitual mailbox for example... The simplest way to do that would be to get VoIP phone service from a provider like Vonage and the simply pack up the VoIP adapter and US phone and install it wherever we are - using the proper power converters. However, we are doing something slightly more convoluted. Even though Skype became famous for offering free computer to computer calling using a peer-peer, the now also offer gateways to the plain-old telephone service at a fraction of the cost of most other VoIP providers. For $9 per month, we have unlimited callin

Linking checkins to tickets (SDI 07 VII)

There are two crucial pieces information related to every change to the project's codebase: the what and the why. Thanks to using svn for version control, all the details regarding what a change is are being automatically recorded in the form of an atomic changeset which moves the codebase from one (hopefully) consistent state to another. Recording the why requires a bit more work. In this simple workflow, we are using Track tickets to document and track every single legitimate reason for making a change to the codebase - whether they are bugfixes or work items related to new features or enhancements. If every legitimate reason to change the code is represented by a ticket, then for each svn changeset there should be one or more tickets which shows why this change was made. Code changes shoudl be done in small incremental steps, but the reason for these changes may be a fairly large an complicated project requiring many small code changes. Tickets can also be used as threads for re

Virtual Mail

We are about to move to Europe for a few years and during that time, we need to be able to maintain a virtual presence here in the US, to make going back and forth more easy, to stay in touch with friends and family and to take advantage of the greatest consumer paradise on earth, where everything is available for a buck or two. An important part of maintaining a presence is to have a US mailing address - e.g. this required to have a US credit card and to have stuff shipped to you. For certain things a PO box would do, for others you need a real street address - e.g. to accept packages. Traditional mail forwarding places have existed for a long time, basically private mailbox operators who also provide the service to periodically mail the content of the mailbox to a forwarding address. Shipping overseas is really expensive and the idea of getting a box of junk-mail sent to us every months or so does not seem too appealing. Also if something is really important, it might also be urgent

Manhattanhenge II

Today was the second occurrence of Manhattanhenge this year - both equally before and after the summer solstice (June 21). I didn't take any pictures this time, largely because it was cloudy and rainy tonight.

Component Architecture, Android Style

Android has a nifty component framework, where each screen - called " activities " should be self-contain can be called up by anybody through an event distribution mechanism called " intents ". Activities can also register to handle arbitrary intents, which allows applications to delete certain functionality without even knowing which app is going to provide this functionality. Certain intents are pre-defined - e.g. the NoiseAlert app uses the ACTION_CALL system intent to delegate the making of a phone call to the dialer app typically, or whoever can handle it. Anybody can define new intents, but to pull off useful delegation of functionality between completely unrelated apps is usually a bit harder to pull off. But here is an example of that. A while ago, a user of the BistroMath tip calculator was asking whether I could add the functionality of recording and tracking dining expenses over time. I can see this would be a very useful functionality. But I also thin

Androlib

As the number of apps on the Android Market is growing, finding interesting apps is increasingly becoming harder. Developers have a very limited opportunity to promote their apps - a name, an icon and 325 characters are all there is to tell the user what the app is about. No screen-shots, no release notes, no FAQ, no back-channel to respond to user's comments. And worst of all, there is still no web interface which puts whatever little information there is on the market online and available to search engines. However useful smart phones are supposed to be the speed, power and comfort of a full-size laptop or desktop computer are hard to beat when it comes to searching and browsing large amounts of data. The quality of the built-in search function in the Android market is a bit spotty to say the least - why not let people use real powertools like an Internet search engine to find android apps? For a while there has been Cyrket to expose the content of the market online - presumably

E-mail notifications for SVN & Trac (SDI 07 part VI)

Much of the email I get is actually not from users directly, but generated by some automated system in response to some event. We are all used to getting these kind of emails - if we sign up on a new Internet sit, buy tickets online or somebody throws a sheep at us on Facebook . Since email is nearly ubiquitous and has many powerful tools for sorting, filtering, archiving or processing messages, it is a natural channel for delivering such automated notifications. Since we set up an email distribution system in the last 2 episodes of our series on startup software development infrastructure, we can make use of it as well to distribute event notifications from the key systems in our software development workflow, which are the issue tracking and source code version control systems - Trac tickets and svn respectively in our case. To enable email notification for every change to a trac ticket which concerns us, we only need to edit the notification section in the Trac config file at /dat

Archived Mailing Lists (SDI 07 Part V)

In the previous episode of our series on startup software development infrastructure, we have setup the basic email deliver system. The next step is to provide a solution for archived mailing lists. Mailing lists have long been a backbone of the Internet community and there are mature solutions for managing very large-scale mailing lists. The classics are Listserv and Majordomo where users can manage their own subscriptions by sending emails with embedded commands and more recently Mailman has become very popular because of its web interface for managing mailing list settings for both users and administrators. Mailman even has a built-in web-based email archive. Since we are targeting our solution for very small teams, we can get away with an even simpler approach. We expect our teams to only have a handful of members and where people joining and leaving are very rare events. In this case, we can simply use the mail alias functionality built into the email server. To maintain th

Local Email System (SDI 07 Part IV)

After setting up Subversion and Trac for managing source code, the next episode in our series on startup software development infrastructure is about email. Email or some other form of archived group communication is essential for a team to remain in sync on any of the important details of the project. Using email for remote collaboration is about as old as the Internet itself and benefits from well established habits and usage pattern. Most open source projects are using mailing lists as their main or sole channel of communication, which means that most open-source software development tools are well integrated with an email centric work flow. Systematically conducting all important technical discussions on archived mailing lists can bridge gaps in both space and time. Email can reach team members who are not here right now - traveling or in a remote location as well as future team members who can read up on old discussion threads to figure out why things were done a certain way. T

Code Review App as a Service

I just noticed that Rietveld, one of the open-source code-review applications dicussed in an earlier post is now availalbe as a hosted virtual private application to any organization which uses the Google Apps service. Since I use Google Apps for the @kugelfish.com domain, I tried out the new service by adding it to my domain from the link at this page . As for all virtualized apps, the administrator needs to create a new CNAME in the domains DNS records under which the new service will be mapped - e.g. reviews.kugelfish.com. While there are many good arguments for an organization to use a hosted solution like Google Apps for providing their public email service, I am a bit more skeptical about outsourcing critical development infrastructure. Specially as in this case the SLA explicitly says that there is no SLA, since these are experimental labs applications - a step down even from the usual Google Beta label. But for organizations where code reviews are not yet a fully integrated a

Manhattanhenge

Today is a geeky holiday: the particular celestial alignment called Manhattanhenge where the sun sets exactly in line with the Manhattan street grid, which is about 30 degrees off from perfect east-west orientation. This picture which is taken from 5th avenue, shows the sun setting at the end of 22nd Street. Not exactly at street level and somewhat before the astronomical time of sunset, since the mathematical horizon is obstructed the coastline of New Jersey in the background. Some more pictures on flickr from 23rd st and 22nd st .

Poor-man's Testing Cluster

One of the biggest problems in embedded software development is how to most effectively test while the final target hardware is either not built yet or is too rare, unwieldy or expensive to give every software developer unlimited and unrestricted access to. Typical approaches include software simulation of the target on some general purpose computer or to find some suitable stand-in for the final hardware platform. Simulation by running the target OS on another CPU architecture (e.g. PowerPC target on an Intel based PC) is inaccurate for some of the important differences between CPU architectures like endianness and memory alignment . Simulation including the target processor architecture can be very slow unless the speed difference between the target platform (e.g. a low powered mobile device) and the host platform (e.g. a standard desktop PC or server) is sufficiently large to make the experience usable for developers - e.g. for the Android emulator included in the SDK. The often s

Submarine Mode

I understand that one of the Google's ulterior motives with Android is to promote a mobile experience where the user is always connected to the Internet and the G1 is pretty much built around that "always on" networking paradigm - including the special flat-rate data plans from T-Mobile. On the other hand, data services are not universally cheap yet everywhere in the world and it would be nice to give the user more control over the mobile data usage. Both current commercial Android phones (HTC G1/Dream and G2/Magic) have two types of radio for data usage: GPRS/EDGE/3G cellular data connection IEEE 802.11 WiFi wireless LAN interface Since the Wifi interface is faster and was not exactly invented with power saving mobile devices in mind, it is presumably more power hungry than the cellular interface. In the implementation on the G1, Android gives precedence to the Wifi connection if enabled and available when the phone is active - i.e. when the screen is on. Once the screen

Subversion & Trac (SDI 07 Part III)

In this episde of the series on creating a minimal software development infrastructure, we are dealing with the centerpiece of the solution: setting up Subversion as the version control system and Trac to provide a unified and integrated system for bug/issue tracking, collaborative document editing (wiki) and source control repository browsing as well as a platform for further integration and extension . All the necessary packages are already installed on the server as part of the previous episode . Assuming that our infrastructure server has a payload partition mounted under /data and our fictitious project is called "sdi07", we are setting up the disk space for our project as follows: mkdir /data/ mkdir /data/sdi07/svn mkdir /data/sdi07/trac To properly initialize the databases for svn and trac, we need to run the following commands, svnadmin create /data/sdi07/svn trac-admin /data/sdi07/trac initenv and answer a few basic questions for trac-admin . In particular we n

Platform Setup (SDI 07 Part II)

In the last episode of this series , we have decided to use Gentoo Linux on a skimpy tabletop server as the platform for the software development infrastructure for our fictitious new project. True to the hardcore image of Gentoo, the installation process for the Gentoo bootstrap binary distribution is a bit spartan (as of 2007). Fortunately Sabayon Linux provides a Gentoo derived distribution with a live CD to check out hardware compatibility and a simple installation process targeted at desktop end-users. Ok, I desktop distribution is probably not optimal for a server, but I needed to get the base system up and running as painlessly as possible (which it did). Once the minimal base system is configured to connect to the local network and is ready for remote login by an admin user, we can start with the setup of the service infrastructure. Based on the list of the services we want to set up, this is the shopping list of additional packages, which we need to download from the portag

Cupcake is out of the Oven

The new version of the Android platform - 1.5 "Cupcake" - is now being shipped with the new HTC Magic phone from Vodafone and is also already available for some versions of the HTC Dream/G1. Since an OS update in the field is aways a scary business - T-Mobile is likely going to take it slow to upgrade all of the reportedly over a million sold G1 phones. Cupcake seems to be a relatively minor major release - a few significant new features (on-screen keyboard, video), some UI face lift and some improvement behind the scenes (battery life, performance). For my own use, there are two features which have made the upgrade to Cupcake a big deal for me. The touch-screen virtual keyboard is the big one. I have never been a big fan of the G1's hybrid touch-screen plus keyboard design and the virtual keyboard is more than good enough for me. In fact it is a lot better, since I for small text input, work flow from touch-screen navigation to text entry and back is a lot quicker a

Choosing the Operating System (SDI 07 part I)

For the first part of the discussion on how to set up a minimal software development infrastructure for a startup project , using only open-source software, we are looking at the lowest layer in the technology stack - hardware and operating system. The first obvious reason for choosing an operating system for this development support server would be familiarity. If there is a particular OS or distribution the administrator is most familiar and comfortable with - this should probably be the most significant argument for choosing it. At the time of this experiment, I did not have any recent experience with any particular OS for the last few years, so the choice would be based on what I could most likely set up most easily without much of a learning curve and where I could get help most easily when I run into problems. The most obvious choice for an open-source operating system at this point is Linux , which runs pretty much on anything with a CPU - including almost any commodity PC hardw

Essential Startup Software Development Infrastructure - 2007 Edition

A while back, I had done some research into what I would do at this point to set up again the basic software development infrastructure for a new startup project - with hindsight and the state of open-source development tools of ca 2007. Again the goal of the experiment is to spend little or no money, use only free and open-source software and end up with a solution which could be set up from scratch in only a few days. Here are the basic choices of the 2007 edition software development infrastructure (to be elaborated in future posts): The entire development support infrastructure should again be able to run on a single machine. For the experiment, I used my home server running Sabayon Linux a more user friendly version of the Gentoo source based Linux distribution. We assume that this machine is behind a firewall and cannot be accessed from the outside. As the version control system, I chose Subversion (svn). Svn is mature, stable, well supported and largely accepted to be the n

Branching is easy, Merging is hard

Merging concurrent or overlapping changes to the same piece of source-code is one of the basic operations to support collaborative development and is supported in some form by most modern version control systems. Merging is required if changes from two different branches have to be reconciled - these could be branches created explicitly or implicitly by two users editing the same file concurrently as in the example below. In the example below, Alice and Bob are both making changes to the same file. After Alice has submitted a new version, Bob needs to merge the changes into his client view before submitting a new version. Typically a merge is required if a branch is being closed, but the current head version (R43 in the example) on the parent branch where it had branched off from earlier is no longer identical to the version from which it was branched off (R42) - i.e. changes have happened in parallel on both the parent and its derived branch. In such cases a 3-way merge operation can

Startups: Technology Execution Play

At the opposite end of the spectrum from the concept and Zeitgeist heavy startups of the web age , is the kind of startup without neither any particularly great new idea, nor a secret new technology, simply doing something which is really hard to do and only very few people would know how. During the late 1990ies, the Internet had been growing by leaps and bounds, requiring a doubling in capacity every couple of months for many types of networks and networking gear was constantly running out of steam and needed to be upgraded with the next generation of higher capacity equipment. So just building the next bigger, better gear sounded like a reasonable thing to do, except that it was easier said than done - specially at a breakneck pace of 18-24 months development cycles, barely ahead of Moore's law . Until recently, building telecom and networking equipment had been a relatively specialized niche craft, practiced mostly in the R&D labs of a small number of companies, selling to

On the Value of Tools

Maybe to a fault, I tend to think that tools play a big role in the success of a software development projects. The benefits can largely be summarized under the following categories: leverage or force multiplication positive reinforcement or behavioral modification The first one is the primary reason for using tools ever since early hominids started to pick up rocks or sticks and using them as tools. They allow us to go beyond the immediate capacity of our hands or our brains. Even though according to hacker folklore, Real Programmers need nothing but cat > a.out to write code, but the days of writing programs in raw binary form by flipping switches or by punching cards are over. High-level languages and interactive programming - i.e. using a computer workstation to write, compile and test programs in quick iterations, have brought such a leap in programmer productivity, that without it, we could hardly manage the complexity of some of the software systems we are working on today.

Essential Startup Software Development Infrastructure - 2000 Edition

When we started a company in early days of 2000, I spent some time setting up what should become our minimal IT infrastructure and software development environment (That's how I ended up with UID 500...). Since we did not have any money (yet), it had to be free/open-source software, and since we did not have any time for evaluation or in-depth research, we tried to go with what seemed to be the most obvious, conservative or mainstream choice at the time for each piece of the solution. Initially our entire server infrastructure was based on a single Linux box from Penguin Computing since that was about all we could afford with an empty bank account. In the hope there would soon be more machines to come, it was running a NIS and NFS server for a centralized network wide login, DHCP and DNS (bind) servers for IP network configuration, a http server (apache) as the intranet homepage and SMTP (sendmail), POP and IMAP servers for basic email service. Many of these initial choices were

FIRST robotics competition

I was volunteering today at a robot competition for high-school age kids organized by FIRST , a non-profit to promote interest in science and engineering among high-school students. They organize a series of robotics tournaments, where teams of middle-school or high-school age students have to build a robot in 6 weeks to compete in a particular challenge. The teams work with adult mentors, who are typically real-life engineers or scientists. I was impressed by the quality of the work the students brought to today's NY regional competition at the Javits convention center. Most of the robots where highly functional and held up well through multiple rounds of competition. With the disappearance of the industrial middle-class in the US, education has become the single biggest factor in economic success (other than simply being born rich). The service economy consists of at one end of the spectrum of gold-color jobs which typically require advance college degrees and McJobs at the other

SMS Remote Control for Android Apps

I wanted to add an remote control feature to the NoiseAlert application for Android, where menu options could be triggered remotely by sending an SMS to the phone. SMS messages should be delivered to the application only if it is running and the commands should be executed by the foreground activity. Instead of registering a BroadcastReceiver globally in the AndroidManifest.xml file, the following object can dynamically register and unregister itself for receiving all SMS during the time it is active. All incoming SMS are passed to the objects onReceive method, encoded as an Intent in slightly obscure ways: public class SmsRemote extends BroadcastReceiver { Boolean mActive = false; Context mContext; @Override public void onReceive(Context context, Intent intent) { Bundle bundle = intent.getExtras(); if (bundle == null) return; Object pdus[] = (Object[]) bundle.get("pdus"); for (int n = 0; n < pdus.length; n++) { SmsMessage message = SmsM

Source-Code Samples in Blogger

Blogger makes it a bit hard to include properly formated source-code snippets in postings as it does not have a mode for entering raw pre-formatted text, which should not be molested by any of the further processing and rendering. You can always use the raw HTML edit mode, but then all the all the html and xml-isms have to be escaped before pasting in the code sample. Fortunately there is a convenient online service at formatmysourcecode.blogspot.com which does just that. Here is an example of how the resulting output looks: main() { printf("hello, world"); }

The other Benefit of Open-Source

Software development must be one of the fields where the gap between best practices and average practices is the widest. A poll in 2001 showed that only about two thirds of software development teams are using version control and only about one third use some kind of bug tracking system. C'mon people how many high-rise window cleaning crews are working without a safety harness? Open-source projects with many collaborators distributed throughout the world generally need to adopt solid collaborative development practices and often build themselves the tools to support collaboration at such a large scale. With the popularity of open-source software, an increasing number of people in the technical community have been exposed to the ways these projects operate and to the tools they use. Today, it is a lot harder to find fresh college grads who would not find it completely naturally to use version control, after all this is how you get the pre-release version of "insert-your-favori