Start measuring

Category : Android

I have been missing one important part of improving my software for a long time. But after starting my new job at 1&1 in Munich, I was reminded of how important it is.

always measure

Whatever you are trying to build is probably been used by an audience of users. Especially in the mobile sector, most of the software is built for a mass market. Therefore, good quality of an app is a requirement (but not a guarantee) for success. And for improving your software, you want to know which parts users are really using.

Does anyone care about the help document? How many percent of users ever opened it? How much time did they spend reading it? Where are your users from? It is possible to get some information out of Appstore/Market statistics, but only plain download numbers.

If you think measuring only makes sense for high traffic websites, consider this: I am using Google Analytics at this website for some time now. It tracks the location of every user and shows them in a world map. This is accurate enough to show the city of every visitor. With the relatively low number of users, I can have a pretty good guess which citizen of Tartu would visit my homepage. Or if any possible new employer checked out my homepage before the interview. You will find plenty of other interesting questions that can be answered by those numbers.

So head over to the Google Analytics for Mobile website, see how easy it is to integrate into your app and start measuring right away.


Something is happening on the protocol front

Category : networking

For many years the basic layers of network communication have been unquestioned. TCP and HTTP are doing a solid job of delivering static and dynamic content to our browsers. But in some aspects those protocols could do a better job. Especially the TCP Slow Start is noticeable to end users and many big IT companies are cheating on this parameter. In the past, when bandwidth has been scarce, the slow start had less impact on network performance than today. Now, latency becomes the critical parameter, because it makes out for a more and more important part of the delay. Just to give a rough estimate:

The speed of light is 299.792.458 m/s, or around 299.792 km/s. Now let’s assume our webserver is located in California, while the client is from Germany. This is a distance of nearly 10.000 km or a round-trip of 20.000 km. For such a distance, the light could only travel 299.792 / 20.000 = ~ 15 times per second, therefore a round trip takes 67 ms. As long as we can not send data faster than the speed of light, this is our theoretical minimum. A Three Way Handshake can’t be made in less than 100 ms. A brief explanation of this latency problem can be found in the article More Bandwidth Doesn’t Matter (much).

There is a new experimental protocol named SPDY (SPeeDY) trying to get more speed out of HTTP. It is based on some very interesting ideas. Concurrent HTTP connections can be bundled into one SPDY connection, the server is able to notify a client via push and HTTP headers are automatically compressed. On a sidenode, all the traffic is encrypted by SSL. The project overview explains in more detail, how it works and gives some test results.

While SPDY definitely improves the current state, it will not be easy to push a new protocol into the market. The question is, will these improvements just be a bit faster or will they enable a new class of web applications?

Gain productivity by scripting your workflow

Category : shell script

One thing that helps me a lot during my everyday routine is having shortcuts for every simple action I perform. For example

  • uninstalling a specific Android app
  • opening the current XCode project
  • opening a ‘tail -f’ on the logfile of our server

can be reached with two mouseclicks from the dock. Some of those are nothing more than a copy and paste shortcut, the others are one line of shell script. There is really no magic inside, every decent developer should be able to do that. However, at my current workplace, out of ~20 developers, I have never seen anybody creating his own. So take this article as a reminder for reviewing your daily routine and automate the most mundane tasks.

Because creating shortcuts is too easy and doesn’t deserve a howto, I will focus on scripting actions on a remote server. Be sure to use private/public key authentication, so there is no need for typing passwords 20 times a day. This doesn’t require a Unix-style operating system, it is also possible with the Putty Tools on Windows. With that, you can execute remote scripts on a server.

My most often used script looks like this:


Which gives me a logged-in terminal at this server when executed. It is also very helpful to access logfiles with just one click. Just append the remote command to the commandline:

ssh "tail -f /opt/apache-tomcat-6.0.20/logs/catalina.out -n 1000"

That prints the last 1000 lines of the logfile and keeps appending more lines. You might think of many more useful cases already. For example, restarting a server process, watching the network traffic with ngrep or printing the server load. You can also put these scripts into version control and use them with a team of developers.

Scripts like that saved me a lot of time already, and only take a few minutes to set up. And the much more important benefit is, that I use the available information much more often. If it takes 3 seconds to open the logfile, I will look into it even for small problems. But if it takes 30 seconds of a mundane and repetitive workflow…


Android AppRater

Category : Android java

The Android AppRater is a little tool in form of a source code snippet for getting better ratings in the Android Market. Its basic idea is to kindly ask users to rate your application, after they have been using it for a while. Which is a fair deal, because many users only give negative ratings right before uninstalling an application. This way, you only ask those users that are actively using your application.

For my own Android game Laska (light-version) the AppRater is not yet integrated in the current Market version. But it will be part of the next update. The dialog will only be shown if the app is installed for more than three days and has been launched at least seven times. And if you dismiss the dialog, it will not pop up again. This is what it looks like:

Of course, if the application is a piece of crap, this doesn’t help in any way. But for all other apps, it might be worth the effort.

Concurrency in java (german)

Category : Android java

The Free Lunch Is Over – that is the famous headline of an article about how the evolution of hardware alone will not solve our performance problems anymore. With the rise of multicore CPUs, software developers have to put effort into their code, to see further performance gains. Even on mobile phones, as the Tegra 2 shows, we will see more and more multicore CPUs. Therefore, there is a good reason for using concurrency in your application. However, it can also be dangerous and lead to unpredictable errors.


The following is a talk I have given to co-workers about concurrency in general and how to do it in java. It focuses less on academic problems like deadlocks, but shows how painful multi-threading is with the standard library. And there are some best-practices for how to reduce the likeliness of errors.


Building a very easy text classifier in python

Category : python

Some of the developers at match2blue are creating a text-interest-matcher. Leaving buzzword bingo aside, that means the software calculates whether a text is interesting based on users’ interests. So basically you, as a user, have to enter some interests and will be presented some pieces of data in order of their relevance. You can also think of it as text classification into either good or bad.

This software has become quite complicated, because it is necessary to have some kind of semantic knowledge about the interests. But there are different methods of text classification. I was curious about how hard it is to write the most simple text classifier that gives you decent results. Well, turns out it is remarkably simple. Creating the classifier took less than two hours. And here is the source code:

import os
j = os.path.join

def train(text):
	Train a dictionary with the given text. Returns a dictionary of dictionaries,
	that describes the probabilities of all word-folling-ocurrences in the text.

	For example, the string "a test" gives this result:
	>>> train("a test")
	{'': {'a': 1}, 'a': {'test': 1}}
	Meaning that the empty string '' has been followed once by 'a' and 'a' has been
	followed by 'test' as well.

	A longer example leads to a more complex dictionary:
	>>> train("this is a test oh what a test")
	{'': {'this': 1}, 'a': {'test': 2}, 'what': {'a': 1}, 'oh': {'what': 1}, 'this': {'is': 1}, 'is': {'a': 1}, 'test': {'oh': 1}}
	c = {}
	lastword = ""
	for word in text.split():
		word = word.lower()
		if c.has_key(lastword):
			inner = c[lastword]
			if inner.has_key(word):
				inner[word] += 1
				inner[word] = 1
			c[lastword] = {word: 1}
		lastword = word
	return c

def probability_of(dict, lastword, word):
	Helper function for calculating the probability of word following lastword
	in the category given by dict.

	>>> category = train("this is a test")
	>>> probability_of(category, "a", "test")

	>>> probability_of(category, "any", "words")
	word = word.lower()
	if dict.has_key(lastword):
		inner = dict[lastword]
		sumvalues = sum([v for v in inner.values()])
		if inner.has_key(word):
			return inner[word] / (sumvalues * 1.0)
	return 0

def classify(text, dict):
	Returns the probability that a text is from the given category. For every pair of
	words the probability_of value is calculated, summarized and divided by the amount
	of words in the text.

	>>> category = train("this is a test")
	>>> classify("a test with some words", category)

	>>> classify("just writing test or a doesn't improve the ranking", category)
	lastword = ""
	probabilities = 0
	for word in text.split():
		probabilities += probability_of(dict, lastword, word)
		lastword = word
	return probabilities / (len(text.split()) * 1.0)

if __name__ == "__main__":
	Calculate the category, that the text in ../test matches best.
	ranking = []
	for file in os.listdir("categories"):
		trained = train(open(j("categories", file)).read())
		value = classify(open("test").read(), trained)
		print "test is", file, "with", value, "% probability"
		ranking.append((value, file))

	print "The test text is probably from", ranking[-1][1]
	print "(second guess is", ranking[-2][1] + ")"

How does it work? There are two very simple steps it does. First, the classifier has to be trained with existing textfiles. The result is a dictionary that consists of many inner dictionaries. Let’s feed it with some text to see what happens.

In [5]: train("a test a good test a good good test")

{'': {'a': 1},
 'a': {'good': 2, 'test': 1},
 'good': {'good': 1, 'test': 2},
 'test': {'a': 2}}

In the result we can see that three words followed ‘a’. Two times it was ‘good’ and one time ‘test’. This is all we need for classifying. Now we can apply the classify function. It goes through the text to classify and looks for known word follow-ups. If there is a known one, the probability of this follow-up is added. So, in our example the probability of ‘a good’ is 2/3 and for ‘a test’ it is 1/3.

In [2]: a = train("a test a good test a good good test")

In [3]: classify("is it a good test", a)

Out[3]: 0.26666666666666666

In [4]: classify("text good but different style", a)

Out[4]: 0.0

The first example has similar words and ordering as the trained text. The second one also has some exact same words, but they are in a different order. Therefore, the probability of this text beeing a is 0.0.

If you want to try it with some longer text, you can download the classifier from Google Code. It is easy to create your own categories by adding a file in the appropriate folder. Just make sure you have a decent amount of data. Three sentences are not enough for a good classification.

In my tests I got quite good results even for classifying authors writing about the same topic.

cd classify/src/

Things I am missing in Android development

Category : Android

This is a list of things I would like to see for Android to improve the development of apps. Don’t get me wrong, Android is one of the best platforms out there, but this doesn’t mean it can not improve anymore.

Hot Code Replacement
It always takes a while to deploy your application on the mobile phone or in the emulator. If you just want to find out what is happening in the code this deployment cycle adds up to quite some time. I know it is hard to replace Dalvik code from a Sun Java machine, but it would help a lot. We even had hot code replacement with our Robocup machines while they were playing soccer. Another possibility would be to use something like hotswap for python. It monitors changes of source files and applies them to the running process as soon as possible.

Flawless USB Connection
There are many cases where you have to unplug and plugin the development phone again. For example when I adb uninstall an app, it can not be installed again until the USB connection has been reset. The same goes for installing an apk from the SD card. You have to connect the phone as an external device, copy the apk and unplug again, before installing the app.
Another thing is that LogCat most of the time does not recognize when the phone is plugged in again. So you always need to switch to the DDMS view in Eclipse, click on the device and go back to the Java view again.

Simple caching of files
Some time ago I had this question on StackOverflow. Disk space is extremely limited on current Android phones. On the iPhone you can easily bundle 20 Megabytes of data with your application, because the app partition is huge. In contrary, most G1 users don’t even have 3 Megabytes of space on their phones. So until every Android device has a huge app partition (or could use the sdcard for it), we need to cache files and delete them on demand. There should be a standard way of caching, like RomainGuy described it with SoftReferences for storing images in memory.

Just in Time Compiler
Now this one is already in the works, but it might still be a while until it sees the light of day. But obviously the overall performance is not even close to the iPhone right now. A JIT compiler should be able to nearly close the gap.

Hardware Graphics Acceleration
The hardware of most Android devices is capable of handling a lot of 3D objects and textures with high framerates. In the standard user interface, this acceleration is not used. Hardware acceleration with OpenGL should speed up the overall performance a lot.

What kind of things can you think of? Put your own wishlist in the comments.

(this entry is cross-posted from my old blogger site)

Android Fundamentals talk

Category : Android

This is the screencast and voice of my Android Fundamentals talk from december 2009. It was given on the 29th floor of our jentower for the towerbyte.

“Read More”

Android: Deploying multiple targets from one project (outdated)

Category : Android java

Update: This way of deploying multiple targets is considered outdated. There is a better way now.

This posting is about how to create multiple versions of your Android application without cloning the whole project. For example if you want to create a full (paid) app, as well as a lite (free) version of it, you might want to automate the task of switching between them. Both versions should be able to use different graphics, different strings and even different featuresets.

First of all, what causes trouble with multiple targets on Android is the auto-generated source code and the strict checking of Java. Strings and graphics are all kept in one place, namely the res folder. Simply creating one res folder for each target and switch between those folders solves the problem with all resource files. I will give an example Ant-script for this later on.

So, having different resource files seems easy. But there is one more problem. We want to have two different applications, so both targets don’t replace each other on our phone. Meaning, the targets need a unique package name in the AndroidManifest.xml. And it’s getting worse. When changing your applications package name, you also change the package of the automatically generated R file. This R file usually is referenced in a lot of source files – basically everywhere you need graphics, strings or other resources. So you end up editing a great amount of your sourcefiles when changing the application package name.

What is the trick over here? Well, I don’t have any. My approach goes through every Java file and changes the import statement for the R file:

There are several targets in this Ant-script. The default target is myproject, the others depend on the default target (for example myproject is always executed before the otherResources target). myproject does the following:

  • deletes the current res folder
  • copies its own customized resources into res
  • replaces all ocurrences of “import*).R;” with “import;”. This might be necessary if the iamdifferent target changed it to “import;” for example.
  • the package name in the AndroidManifest is changed to the target name

Just execute a target to switch to this version of your app. You need to refresh the project in order to reflect the changes. In Eclipse, this can automatically be activated by setting the following option:

One more thing that does not work out of the box is the android:name parameter in activity declarations of the AndroidManifest. When auto-generated, they are declared relative to the project package. This doesn’t work anymore when the package name is changed. Therefore you have to set the activity name with absolute values. Instead of

<activity name=”.Test”>

You have to write something like

<activity name=””>

For all of you who like free lunch, here is an example project with two targets ready for download.

This is how three targets of the same app look like:

Thats it. With this approach you can deploy as many customized versions of the same project as you want. If you are missing a step or know an optimization of this, please leave me a comment.

(this entry is cross-posted from my old blogger site)

Recent Posts