Google Play Optimization

Currently, I am preparing a presentation about optimizing apps for the Google Play Store. It is an interesting topic and still in its early stages. Unlike SEO for websites, it doesn’t seem to be affected by some page-rank algorithm. Instead the main focus is on keywords and, of course, the ratings of an application.

There is a series of articles about all the optimization topics on droid-blog.net. I highly recommend reading them.

The strong keyword-dependency sometimes leads to unexpected search results. My game laska, for example, is the online strategy game champion (of the world):

Posted in Android | No Comments »

Multiple targets from one Android source (the better way)

Some of you might have read my article Android: Deploying multiple targets from one project. It describes how to create customized versions of the same software and therefore benefit from multiple apps with the same featureset. That deployment with an Ant script has proven to work well. For example our GMX Mail App is available in four different customizations, for different brands, and uses a similar approach with maven.

However, there is a better way now to handle multiple targets. It is less complex and gives you even more options to customize the different targets. By using an Android Library Project, you still have the benefit of sharing resources and code, without the hacky Ant script. Remember, the Ant script would go through every Java source file and change an import statement, just to resolve the different package name of the R File. Switching between Targets required an Ant build with a refresh of the workspace. Not any more. Now switching between projects is as simple as clicking the run button in Eclipse. Especially for bigger projects this is a huge benefit, because refreshing the workspace can take quite some time.

So what is the new setup? You need to create one base project, with everything common inside, and declare it as an Android Library Project. This option is available under project properties in the Android tab. Then you create the first one of your targets as a different Android Project in Eclipse. On the same properties tab of the new project, you add the base project as a library. Repeat it for another project, which will be your second target. Now you will have somthing similar to this:

For the showcase, I deleted all source files of the custom projects. Since we want to re-use the majority of our sourcecode from the base project, we don’t need any custom-sources right now. There is one little fix we need to apply to the AndroidManifest.xml file. The Android Wizard in Eclipse uses relative references to our Activities. This does not work if we want to use our Activity from the base project, because it uses a different package than our custom project. Therefore, we have to specify the full package name to the specific Activity. In my sample the important part looks like this:

That’s it. You can now overwrite all your base resources and source files in the custom projects. Every new feature developed in the base project is immediately available everywhere. Only if there is a need for updating the AndroidManifest, you have to edit it in all custom projects. But this also means you have a fine grained control over the manifest file.

I updated my old example project on Google Code. Feel free to use it as a start for your own project with multiple targets. Feedback and contributions are always welcome.

Posted in Android, java | 8 Comments »

Key learnings from analytics

Enough time has passed since I put Google Analytics into my Android game Laska. It has been collecting statistics for nearly a month now. Therefore, I want to share some data and show the key learnings I got out of it.

The majority of my users are from China and Poland

This was quite unexpected for me. The game is pretty german. It is even named after Emmanuel Lasker, the famous chess player. However, Germany is responsible for only a very small part of the visits. It is interesting for future localizations, because a mandarin translation might make a lot more sense than a german one.

There is always more to track

I started off with only tracking the in-app pages and a variable for won/lost games. But with this new information, there is already the next pending question. How many games were played at the easy level? How many at hard? What is the percentage of won games on hard? There is always more to measure and I will implement it in future versions of the game.

Devices used

The Samsung Galaxy S and HTC Wildfire are responsible for nearly half the usage. The most common display size is 480 x 800 with 42% of the visits, followed by 240 x 320 with nearly 30%. I did not expect the small display devices to have such a big share of the visits. One reason might be, that users in China and Poland are more likely to use lower end android devices than in the US or Germany.

 

Posted in Android | No Comments »

Start measuring

I have been missing one important part of improving my software for a long time. But after starting my new job at 1&1 in Munich, I was reminded of how important it is.

always measure

Whatever you are trying to build is probably been used by an audience of users. Especially in the mobile sector, most of the software is built for a mass market. Therefore, good quality of an app is a requirement (but not a guarantee) for success. And for improving your software, you want to know which parts users are really using.

Does anyone care about the help document? How many percent of users ever opened it? How much time did they spend reading it? Where are your users from? It is possible to get some information out of Appstore/Market statistics, but only plain download numbers.

If you think measuring only makes sense for high traffic websites, consider this: I am using Google Analytics at this website for some time now. It tracks the location of every user and shows them in a world map. This is accurate enough to show the city of every visitor. With the relatively low number of users, I can have a pretty good guess which citizen of Tartu would visit my homepage. Or if any possible new employer checked out my homepage before the interview. You will find plenty of other interesting questions that can be answered by those numbers.

So head over to the Google Analytics for Mobile website, see how easy it is to integrate into your app and start measuring right away.

 

Posted in Android | No Comments »

Something is happening on the protocol front

For many years the basic layers of network communication have been unquestioned. TCP and HTTP are doing a solid job of delivering static and dynamic content to our browsers. But in some aspects those protocols could do a better job. Especially the TCP Slow Start is noticeable to end users and many big IT companies are cheating on this parameter. In the past, when bandwidth has been scarce, the slow start had less impact on network performance than today. Now, latency becomes the critical parameter, because it makes out for a more and more important part of the delay. Just to give a rough estimate:

The speed of light is 299.792.458 m/s, or around 299.792 km/s. Now let’s assume our webserver is located in California, while the client is from Germany. This is a distance of nearly 10.000 km or a round-trip of 20.000 km. For such a distance, the light could only travel 299.792 / 20.000 = ~ 15 times per second, therefore a round trip takes 67 ms. As long as we can not send data faster than the speed of light, this is our theoretical minimum. A Three Way Handshake can’t be made in less than 100 ms. A brief explanation of this latency problem can be found in the article More Bandwidth Doesn’t Matter (much).

There is a new experimental protocol named SPDY (SPeeDY) trying to get more speed out of HTTP. It is based on some very interesting ideas. Concurrent HTTP connections can be bundled into one SPDY connection, the server is able to notify a client via push and HTTP headers are automatically compressed. On a sidenode, all the traffic is encrypted by SSL. The project overview explains in more detail, how it works and gives some test results.

While SPDY definitely improves the current state, it will not be easy to push a new protocol into the market. The question is, will these improvements just be a bit faster or will they enable a new class of web applications?

Posted in networking | No Comments »

Gain productivity by scripting your workflow

One thing that helps me a lot during my everyday routine is having shortcuts for every simple action I perform. For example

  • uninstalling a specific Android app
  • opening the current XCode project
  • opening a ‘tail -f’ on the logfile of our server

can be reached with two mouseclicks from the dock. Some of those are nothing more than a copy and paste shortcut, the others are one line of shell script. There is really no magic inside, every decent developer should be able to do that. However, at my current workplace, out of ~20 developers, I have never seen anybody creating his own. So take this article as a reminder for reviewing your daily routine and automate the most mundane tasks.

Because creating shortcuts is too easy and doesn’t deserve a howto, I will focus on scripting actions on a remote server. Be sure to use private/public key authentication, so there is no need for typing passwords 20 times a day. This doesn’t require a Unix-style operating system, it is also possible with the Putty Tools on Windows. With that, you can execute remote scripts on a server.

My most often used script looks like this:

ssh root@myserver.com

Which gives me a logged-in terminal at this server when executed. It is also very helpful to access logfiles with just one click. Just append the remote command to the commandline:

ssh tomcat@myserver.com "tail -f /opt/apache-tomcat-6.0.20/logs/catalina.out -n 1000"

That prints the last 1000 lines of the logfile and keeps appending more lines. You might think of many more useful cases already. For example, restarting a server process, watching the network traffic with ngrep or printing the server load. You can also put these scripts into version control and use them with a team of developers.

Scripts like that saved me a lot of time already, and only take a few minutes to set up. And the much more important benefit is, that I use the available information much more often. If it takes 3 seconds to open the logfile, I will look into it even for small problems. But if it takes 30 seconds of a mundane and repetitive workflow…

 

Posted in shell script | No Comments »

Android AppRater

The Android AppRater is a little tool in form of a source code snippet for getting better ratings in the Android Market. Its basic idea is to kindly ask users to rate your application, after they have been using it for a while. Which is a fair deal, because many users only give negative ratings right before uninstalling an application. This way, you only ask those users that are actively using your application.

For my own Android game Laska (light-version) the AppRater is not yet integrated in the current Market version. But it will be part of the next update. The dialog will only be shown if the app is installed for more than three days and has been launched at least seven times. And if you dismiss the dialog, it will not pop up again. This is what it looks like:

Of course, if the application is a piece of crap, this doesn’t help in any way. But for all other apps, it might be worth the effort.

Posted in Android, java | No Comments »

Concurrency in java (german)

The Free Lunch Is Over – that is the famous headline of an article about how the evolution of hardware alone will not solve our performance problems anymore. With the rise of multicore CPUs, software developers have to put effort into their code, to see further performance gains. Even on mobile phones, as the Tegra 2 shows, we will see more and more multicore CPUs. Therefore, there is a good reason for using concurrency in your application. However, it can also be dangerous and lead to unpredictable errors.

 

The following is a talk I have given to co-workers about concurrency in general and how to do it in java. It focuses less on academic problems like deadlocks, but shows how painful multi-threading is with the standard library. And there are some best-practices for how to reduce the likeliness of errors.

 

Posted in Android, java | No Comments »

Building a very easy text classifier in python

Some of the developers at match2blue are creating a text-interest-matcher. Leaving buzzword bingo aside, that means the software calculates whether a text is interesting based on users’ interests. So basically you, as a user, have to enter some interests and will be presented some pieces of data in order of their relevance. You can also think of it as text classification into either good or bad.

This software has become quite complicated, because it is necessary to have some kind of semantic knowledge about the interests. But there are different methods of text classification. I was curious about how hard it is to write the most simple text classifier that gives you decent results. Well, turns out it is remarkably simple. Creating the classifier took less than two hours. And here is the source code:

import os
j = os.path.join

def train(text):
	"""
	Train a dictionary with the given text. Returns a dictionary of dictionaries,
	that describes the probabilities of all word-folling-ocurrences in the text.

	For example, the string "a test" gives this result:
	>>> train("a test")
	{'': {'a': 1}, 'a': {'test': 1}}
	Meaning that the empty string '' has been followed once by 'a' and 'a' has been
	followed by 'test' as well.

	A longer example leads to a more complex dictionary:
	>>> train("this is a test oh what a test")
	{'': {'this': 1}, 'a': {'test': 2}, 'what': {'a': 1}, 'oh': {'what': 1}, 'this': {'is': 1}, 'is': {'a': 1}, 'test': {'oh': 1}}
	"""
	c = {}
	lastword = ""
	for word in text.split():
		word = word.lower()
		if c.has_key(lastword):
			inner = c[lastword]
			if inner.has_key(word):
				inner[word] += 1
			else:
				inner[word] = 1
		else:
			c[lastword] = {word: 1}
		lastword = word
	return c

def probability_of(dict, lastword, word):
	"""
	Helper function for calculating the probability of word following lastword
	in the category given by dict.

	>>> category = train("this is a test")
	>>> probability_of(category, "a", "test")
	1.0

	>>> probability_of(category, "any", "words")
	0
	"""
	word = word.lower()
	if dict.has_key(lastword):
		inner = dict[lastword]
		sumvalues = sum([v for v in inner.values()])
		if inner.has_key(word):
			return inner[word] / (sumvalues * 1.0)
	return 0

def classify(text, dict):
	"""
	Returns the probability that a text is from the given category. For every pair of
	words the probability_of value is calculated, summarized and divided by the amount
	of words in the text.

	>>> category = train("this is a test")
	>>> classify("a test with some words", category)
	0.2

	>>> classify("just writing test or a doesn't improve the ranking", category)
	0.0
	"""
	lastword = ""
	probabilities = 0
	for word in text.split():
		probabilities += probability_of(dict, lastword, word)
		lastword = word
	return probabilities / (len(text.split()) * 1.0)

if __name__ == "__main__":
	"""
	Calculate the category, that the text in ../test matches best.
	"""
	ranking = []
	os.chdir("..")
	for file in os.listdir("categories"):
		trained = train(open(j("categories", file)).read())
		value = classify(open("test").read(), trained)
		print "test is", file, "with", value, "% probability"
		ranking.append((value, file))

	ranking.sort()
	print
	print "The test text is probably from", ranking[-1][1]
	print "(second guess is", ranking[-2][1] + ")"

How does it work? There are two very simple steps it does. First, the classifier has to be trained with existing textfiles. The result is a dictionary that consists of many inner dictionaries. Let’s feed it with some text to see what happens.

In [5]: train("a test a good test a good good test")

Out[5]:
{'': {'a': 1},
 'a': {'good': 2, 'test': 1},
 'good': {'good': 1, 'test': 2},
 'test': {'a': 2}}

In the result we can see that three words followed ‘a’. Two times it was ‘good’ and one time ‘test’. This is all we need for classifying. Now we can apply the classify function. It goes through the text to classify and looks for known word follow-ups. If there is a known one, the probability of this follow-up is added. So, in our example the probability of ‘a good’ is 2/3 and for ‘a test’ it is 1/3.

In [2]: a = train("a test a good test a good good test")

In [3]: classify("is it a good test", a)

Out[3]: 0.26666666666666666

In [4]: classify("text good but different style", a)

Out[4]: 0.0

The first example has similar words and ordering as the trained text. The second one also has some exact same words, but they are in a different order. Therefore, the probability of this text beeing a is 0.0.

If you want to try it with some longer text, you can download the classifier from Google Code. It is easy to create your own categories by adding a file in the appropriate folder. Just make sure you have a decent amount of data. Three sentences are not enough for a good classification.

In my tests I got quite good results even for classifying authors writing about the same topic.

cd classify/src/
python classify.py
Posted in python | No Comments »

Things I am missing in Android development

This is a list of things I would like to see for Android to improve the development of apps. Don’t get me wrong, Android is one of the best platforms out there, but this doesn’t mean it can not improve anymore.

Hot Code Replacement
It always takes a while to deploy your application on the mobile phone or in the emulator. If you just want to find out what is happening in the code this deployment cycle adds up to quite some time. I know it is hard to replace Dalvik code from a Sun Java machine, but it would help a lot. We even had hot code replacement with our Robocup machines while they were playing soccer. Another possibility would be to use something like hotswap for python. It monitors changes of source files and applies them to the running process as soon as possible.

Flawless USB Connection
There are many cases where you have to unplug and plugin the development phone again. For example when I adb uninstall an app, it can not be installed again until the USB connection has been reset. The same goes for installing an apk from the SD card. You have to connect the phone as an external device, copy the apk and unplug again, before installing the app.
Another thing is that LogCat most of the time does not recognize when the phone is plugged in again. So you always need to switch to the DDMS view in Eclipse, click on the device and go back to the Java view again.

Simple caching of files
Some time ago I had this question on StackOverflow. Disk space is extremely limited on current Android phones. On the iPhone you can easily bundle 20 Megabytes of data with your application, because the app partition is huge. In contrary, most G1 users don’t even have 3 Megabytes of space on their phones. So until every Android device has a huge app partition (or could use the sdcard for it), we need to cache files and delete them on demand. There should be a standard way of caching, like RomainGuy described it with SoftReferences for storing images in memory.

Just in Time Compiler
Now this one is already in the works, but it might still be a while until it sees the light of day. But obviously the overall performance is not even close to the iPhone right now. A JIT compiler should be able to nearly close the gap.

Hardware Graphics Acceleration
The hardware of most Android devices is capable of handling a lot of 3D objects and textures with high framerates. In the standard user interface, this acceleration is not used. Hardware acceleration with OpenGL should speed up the overall performance a lot.

What kind of things can you think of? Put your own wishlist in the comments.

(this entry is cross-posted from my old blogger site)

Posted in Android | No Comments »