kugelfish

Much of the email I get is actually not from users directly, but generated by some automated system in response to some event. We are all used to getting these kind of emails - if we sign up on a new Internet sit, buy tickets online or somebody throws a sheep at us on Facebook . Since email is nearly ubiquitous and has many powerful tools for sorting, filtering, archiving or processing messages, it is a natural channel for delivering such automated notifications. Since we set up an email distribution system in the last 2 episodes of our series on startup software development infrastructure, we can make use of it as well to distribute event notifications from the key systems in our software development workflow, which are the issue tracking and source code version control systems - Trac tickets and svn respectively in our case. To enable email notification for every change to a trac ticket which concerns us, we only need to edit the notification section in the Trac config file at /dat...

In the previous episode of our series on startup software development infrastructure, we have setup the basic email deliver system. The next step is to provide a solution for archived mailing lists. Mailing lists have long been a backbone of the Internet community and there are mature solutions for managing very large-scale mailing lists. The classics are Listserv and Majordomo where users can manage their own subscriptions by sending emails with embedded commands and more recently Mailman has become very popular because of its web interface for managing mailing list settings for both users and administrators. Mailman even has a built-in web-based email archive. Since we are targeting our solution for very small teams, we can get away with an even simpler approach. We expect our teams to only have a handful of members and where people joining and leaving are very rare events. In this case, we can simply use the mail alias functionality built into the email server. To maintain th...

After setting up Subversion and Trac for managing source code, the next episode in our series on startup software development infrastructure is about email. Email or some other form of archived group communication is essential for a team to remain in sync on any of the important details of the project. Using email for remote collaboration is about as old as the Internet itself and benefits from well established habits and usage pattern. Most open source projects are using mailing lists as their main or sole channel of communication, which means that most open-source software development tools are well integrated with an email centric work flow. Systematically conducting all important technical discussions on archived mailing lists can bridge gaps in both space and time. Email can reach team members who are not here right now - traveling or in a remote location as well as future team members who can read up on old discussion threads to figure out why things were done a certain way. T...

I just noticed that Rietveld, one of the open-source code-review applications dicussed in an earlier post is now availalbe as a hosted virtual private application to any organization which uses the Google Apps service. Since I use Google Apps for the @kugelfish.com domain, I tried out the new service by adding it to my domain from the link at this page . As for all virtualized apps, the administrator needs to create a new CNAME in the domains DNS records under which the new service will be mapped - e.g. reviews.kugelfish.com. While there are many good arguments for an organization to use a hosted solution like Google Apps for providing their public email service, I am a bit more skeptical about outsourcing critical development infrastructure. Specially as in this case the SLA explicitly says that there is no SLA, since these are experimental labs applications - a step down even from the usual Google Beta label. But for organizations where code reviews are not yet a fully integrated a...

Today is a geeky holiday: the particular celestial alignment called Manhattanhenge where the sun sets exactly in line with the Manhattan street grid, which is about 30 degrees off from perfect east-west orientation. This picture which is taken from 5th avenue, shows the sun setting at the end of 22nd Street. Not exactly at street level and somewhat before the astronomical time of sunset, since the mathematical horizon is obstructed the coastline of New Jersey in the background. Some more pictures on flickr from 23rd st and 22nd st .

One of the biggest problems in embedded software development is how to most effectively test while the final target hardware is either not built yet or is too rare, unwieldy or expensive to give every software developer unlimited and unrestricted access to. Typical approaches include software simulation of the target on some general purpose computer or to find some suitable stand-in for the final hardware platform. Simulation by running the target OS on another CPU architecture (e.g. PowerPC target on an Intel based PC) is inaccurate for some of the important differences between CPU architectures like endianness and memory alignment . Simulation including the target processor architecture can be very slow unless the speed difference between the target platform (e.g. a low powered mobile device) and the host platform (e.g. a standard desktop PC or server) is sufficiently large to make the experience usable for developers - e.g. for the Android emulator included in the SDK. The often s...

I understand that one of the Google's ulterior motives with Android is to promote a mobile experience where the user is always connected to the Internet and the G1 is pretty much built around that "always on" networking paradigm - including the special flat-rate data plans from T-Mobile. On the other hand, data services are not universally cheap yet everywhere in the world and it would be nice to give the user more control over the mobile data usage. Both current commercial Android phones (HTC G1/Dream and G2/Magic) have two types of radio for data usage: GPRS/EDGE/3G cellular data connection IEEE 802.11 WiFi wireless LAN interface Since the Wifi interface is faster and was not exactly invented with power saving mobile devices in mind, it is presumably more power hungry than the cellular interface. In the implementation on the G1, Android gives precedence to the Wifi connection if enabled and available when the phone is active - i.e. when the screen is on. Once the screen...

In this episde of the series on creating a minimal software development infrastructure, we are dealing with the centerpiece of the solution: setting up Subversion as the version control system and Trac to provide a unified and integrated system for bug/issue tracking, collaborative document editing (wiki) and source control repository browsing as well as a platform for further integration and extension . All the necessary packages are already installed on the server as part of the previous episode . Assuming that our infrastructure server has a payload partition mounted under /data and our fictitious project is called "sdi07", we are setting up the disk space for our project as follows: mkdir /data/ mkdir /data/sdi07/svn mkdir /data/sdi07/trac To properly initialize the databases for svn and trac, we need to run the following commands, svnadmin create /data/sdi07/svn trac-admin /data/sdi07/trac initenv and answer a few basic questions for trac-admin . In particular we n...

In the last episode of this series , we have decided to use Gentoo Linux on a skimpy tabletop server as the platform for the software development infrastructure for our fictitious new project. True to the hardcore image of Gentoo, the installation process for the Gentoo bootstrap binary distribution is a bit spartan (as of 2007). Fortunately Sabayon Linux provides a Gentoo derived distribution with a live CD to check out hardware compatibility and a simple installation process targeted at desktop end-users. Ok, I desktop distribution is probably not optimal for a server, but I needed to get the base system up and running as painlessly as possible (which it did). Once the minimal base system is configured to connect to the local network and is ready for remote login by an admin user, we can start with the setup of the service infrastructure. Based on the list of the services we want to set up, this is the shopping list of additional packages, which we need to download from the portag...

The new version of the Android platform - 1.5 "Cupcake" - is now being shipped with the new HTC Magic phone from Vodafone and is also already available for some versions of the HTC Dream/G1. Since an OS update in the field is aways a scary business - T-Mobile is likely going to take it slow to upgrade all of the reportedly over a million sold G1 phones. Cupcake seems to be a relatively minor major release - a few significant new features (on-screen keyboard, video), some UI face lift and some improvement behind the scenes (battery life, performance). For my own use, there are two features which have made the upgrade to Cupcake a big deal for me. The touch-screen virtual keyboard is the big one. I have never been a big fan of the G1's hybrid touch-screen plus keyboard design and the virtual keyboard is more than good enough for me. In fact it is a lot better, since I for small text input, work flow from touch-screen navigation to text entry and back is a lot quicker a...

For the first part of the discussion on how to set up a minimal software development infrastructure for a startup project , using only open-source software, we are looking at the lowest layer in the technology stack - hardware and operating system. The first obvious reason for choosing an operating system for this development support server would be familiarity. If there is a particular OS or distribution the administrator is most familiar and comfortable with - this should probably be the most significant argument for choosing it. At the time of this experiment, I did not have any recent experience with any particular OS for the last few years, so the choice would be based on what I could most likely set up most easily without much of a learning curve and where I could get help most easily when I run into problems. The most obvious choice for an open-source operating system at this point is Linux , which runs pretty much on anything with a CPU - including almost any commodity PC hardw...

A while back, I had done some research into what I would do at this point to set up again the basic software development infrastructure for a new startup project - with hindsight and the state of open-source development tools of ca 2007. Again the goal of the experiment is to spend little or no money, use only free and open-source software and end up with a solution which could be set up from scratch in only a few days. Here are the basic choices of the 2007 edition software development infrastructure (to be elaborated in future posts): The entire development support infrastructure should again be able to run on a single machine. For the experiment, I used my home server running Sabayon Linux a more user friendly version of the Gentoo source based Linux distribution. We assume that this machine is behind a firewall and cannot be accessed from the outside. As the version control system, I chose Subversion (svn). Svn is mature, stable, well supported and largely accepted to be the n...

Merging concurrent or overlapping changes to the same piece of source-code is one of the basic operations to support collaborative development and is supported in some form by most modern version control systems. Merging is required if changes from two different branches have to be reconciled - these could be branches created explicitly or implicitly by two users editing the same file concurrently as in the example below. In the example below, Alice and Bob are both making changes to the same file. After Alice has submitted a new version, Bob needs to merge the changes into his client view before submitting a new version. Typically a merge is required if a branch is being closed, but the current head version (R43 in the example) on the parent branch where it had branched off from earlier is no longer identical to the version from which it was branched off (R42) - i.e. changes have happened in parallel on both the parent and its derived branch. In such cases a 3-way merge operation can ...

At the opposite end of the spectrum from the concept and Zeitgeist heavy startups of the web age , is the kind of startup without neither any particularly great new idea, nor a secret new technology, simply doing something which is really hard to do and only very few people would know how. During the late 1990ies, the Internet had been growing by leaps and bounds, requiring a doubling in capacity every couple of months for many types of networks and networking gear was constantly running out of steam and needed to be upgraded with the next generation of higher capacity equipment. So just building the next bigger, better gear sounded like a reasonable thing to do, except that it was easier said than done - specially at a breakneck pace of 18-24 months development cycles, barely ahead of Moore's law . Until recently, building telecom and networking equipment had been a relatively specialized niche craft, practiced mostly in the R&D labs of a small number of companies, selling to ...

Maybe to a fault, I tend to think that tools play a big role in the success of a software development projects. The benefits can largely be summarized under the following categories: leverage or force multiplication positive reinforcement or behavioral modification The first one is the primary reason for using tools ever since early hominids started to pick up rocks or sticks and using them as tools. They allow us to go beyond the immediate capacity of our hands or our brains. Even though according to hacker folklore, Real Programmers need nothing but cat > a.out to write code, but the days of writing programs in raw binary form by flipping switches or by punching cards are over. High-level languages and interactive programming - i.e. using a computer workstation to write, compile and test programs in quick iterations, have brought such a leap in programmer productivity, that without it, we could hardly manage the complexity of some of the software systems we are working on today. ...

When we started a company in early days of 2000, I spent some time setting up what should become our minimal IT infrastructure and software development environment (That's how I ended up with UID 500...). Since we did not have any money (yet), it had to be free/open-source software, and since we did not have any time for evaluation or in-depth research, we tried to go with what seemed to be the most obvious, conservative or mainstream choice at the time for each piece of the solution. Initially our entire server infrastructure was based on a single Linux box from Penguin Computing since that was about all we could afford with an empty bank account. In the hope there would soon be more machines to come, it was running a NIS and NFS server for a centralized network wide login, DHCP and DNS (bind) servers for IP network configuration, a http server (apache) as the intranet homepage and SMTP (sendmail), POP and IMAP servers for basic email service. Many of these initial choices were ...

I was volunteering today at a robot competition for high-school age kids organized by FIRST , a non-profit to promote interest in science and engineering among high-school students. They organize a series of robotics tournaments, where teams of middle-school or high-school age students have to build a robot in 6 weeks to compete in a particular challenge. The teams work with adult mentors, who are typically real-life engineers or scientists. I was impressed by the quality of the work the students brought to today's NY regional competition at the Javits convention center. Most of the robots where highly functional and held up well through multiple rounds of competition. With the disappearance of the industrial middle-class in the US, education has become the single biggest factor in economic success (other than simply being born rich). The service economy consists of at one end of the spectrum of gold-color jobs which typically require advance college degrees and McJobs at the other...

I wanted to add an remote control feature to the NoiseAlert application for Android, where menu options could be triggered remotely by sending an SMS to the phone. SMS messages should be delivered to the application only if it is running and the commands should be executed by the foreground activity. Instead of registering a BroadcastReceiver globally in the AndroidManifest.xml file, the following object can dynamically register and unregister itself for receiving all SMS during the time it is active. All incoming SMS are passed to the objects onReceive method, encoded as an Intent in slightly obscure ways: public class SmsRemote extends BroadcastReceiver { Boolean mActive = false; Context mContext; @Override public void onReceive(Context context, Intent intent) { Bundle bundle = intent.getExtras(); if (bundle == null) return; Object pdus[] = (Object[]) bundle.get("pdus"); for (int n = 0; n < pdus.length; n++) { SmsMessage message = SmsM...

Blogger makes it a bit hard to include properly formated source-code snippets in postings as it does not have a mode for entering raw pre-formatted text, which should not be molested by any of the further processing and rendering. You can always use the raw HTML edit mode, but then all the all the html and xml-isms have to be escaped before pasting in the code sample. Fortunately there is a convenient online service at formatmysourcecode.blogspot.com which does just that. Here is an example of how the resulting output looks: main() { printf("hello, world"); }

Software development must be one of the fields where the gap between best practices and average practices is the widest. A poll in 2001 showed that only about two thirds of software development teams are using version control and only about one third use some kind of bug tracking system. C'mon people how many high-rise window cleaning crews are working without a safety harness? Open-source projects with many collaborators distributed throughout the world generally need to adopt solid collaborative development practices and often build themselves the tools to support collaboration at such a large scale. With the popularity of open-source software, an increasing number of people in the technical community have been exposed to the ways these projects operate and to the tools they use. Today, it is a lot harder to find fresh college grads who would not find it completely naturally to use version control, after all this is how you get the pre-release version of "insert-your-favori...