ZEDRAY

In an earlier post I alluded to the idea of using real usage data to improve my Last Call Widget.  Currently the widget packs three key use cases into a small and simple UI:

• Who called last, when and on what number?
• Show the last caller list.
• Return the last call.

My research question focused on what extra information I should be showing, and what other use cases I should be looking to support (or to support better).

The benefits of using an analytics tool are that I get to define my own events (which are linked to specific use cases), and then aggregate huge amounts of real world usage (anonymous of course) to support my conclusions.

I want to send a big thank you to those of you who opted into the Analytics program (all thirteen thousand of you).  I don’t know who you are, but I endeavour to use the information you supplied to make the Widget more useful.

Widget Size

A noisy minority (i.e. 28%) of usage events were generated by people using the mini 2×1 widget, rather than the larger (and original) 4×1 widget.

Widget usage

For both widgets we see an ongoing trend (observed in an earlier post) of the amount of clicks roughly corresponding to the relative size of each button.  Quick dial mode is enabled for roughly half of call button events, either due to genuine user choice or difficulty in discovery (i.e. feature has to be turned on in the options menu).

Return The Last Call?
Now let’s make the data more interesting by comparing new outgoing calls against recent Widget activity.  This will give us a more accurate idea of what the user intended to do, rather than just report on the popularity of existing buttons.

Return Call Log

Of users who clicked on the Call Log button, two thirds found the number they were looking for in the list of five most recent calls, with a quarter needing to find the number elsewhere.  Interestingly, twelve percent of users went on to call a recent caller back on a *different* contact number, with half of these events being for the most recent caller in the list.

Return Call Log Without Widget
We can also make a similar analysis of new outgoing calls when there has been no recent Widget activity (i.e. the user is not using our app).

Now the “Not in History” section has grown much larger (possibly skewed by the user choosing not to use the Widget?), while the “Different number” section has grown to sixteen percent of all outgoing calls.

Conclusion
The Android UI places a large click penalty on calling a user back on a different number.  Hence current thinking is that the widget should support (a) calling back with different numbers, (b) extended call history to better support feature (a).

(a) Different Numbers

When returning a call from the last caller, a quarter of the time the user will want to call back with a different number.  This supports the argument of adding an intermediate screen asking which number the user should be called on (this can be heavily optimised later).

(b) Extended Call History

The last caller (i.e. first in the list) is the most popular option by far, but the second, third and forth options are also represented as more than edge cases.  Hence there is an argument in favour of creating a taller widget that displays more history items, or creating our own call log history screen to replace the native UI.

This second option would simply be an attempt to correct a perceived flaw in the native UI, by integrating the UI for feature (a) for a longer list of contacts.

Next Stage
The next step is to compare the click penalty vs. actual usage, to test some proposed upgrades for their usefulness.

!! None of this is legal advice !!

Last week was a bad one for Analytics, with the *discovery* that an Android wallpaper application was secretly leaking users personal information to a server in China.  The source later backtracked on that claim, but by then the damage (mainly to the trust in having an open ecosystem for Android applications) was already done.

Google themselves weighed in by posting a list of best practices for handling user data on their blog.

I support the appropriate use of metrics and have blogged about it many times, mainly in relation to my Last Call Widget.  Access to real world data is invaluable for analysing how users are utilising an app (testing work flows, detecting bugs, etc), which eventually results in better software for everybody.

Another example I like, is how Replica Island uses anonymous player “death data” to actively improve level design.

Protect Data By Making It Non-identifiable
I am living in a country with some of the tightest data protection laws in the world. That means that whatever the laws are like where you live (if you have data protection legislation?), you are getting the benefit of the protection offered by the German legal system.

As an Analytics user, I am motivated to anonymise all data involved so I can avoid the tight compliance criteria laid down by law (i.e. data security, restricted processing, not passing through third parties).

Note: To Anonymise, means the data can’t be used to trace a specific identifiable person.  Identifiable data might be a unique customer id, zip code + age + gender combination or just a simple telephone number.  Anything the could potentially qualify, should definitely *not* be collected!

Establish Trust
The user must feel conformable that your application isn’t doing anything it shouldn’t.

This is a difficult balance to make, as most users won’t bother reading your descriptive text and will definitely not reach your privacy policy.  I had a one star comment (since removed) from a user who probably hadn’t installed the Widget, but wanted to make a point that the Market description (at the time) didn’t explain what the “Network communication” permission was for.

In Last Call Widget, I now state that the Internet permission is required for metrics - and that this is optional.

I show an opt-in dialog as soon as the user opens the application for the first time (or tries to install from the home screen).  I expect most users to read this text (although it’s not yet localised) and understand that the Widget will do some data collection.  I should say that this is also a Google Analytics requirement.

For added clarity, I also show the current state of the “Data Collection” setting very prominently on the options screen.

Collect Relevant Data
Make your data relevant to what you are trying to measure.  Doing so tends to be safer for your users and makes your results easier to analyse.

The first version of Last Call Widget measured which buttons were pressed and how often, with extra events added later on to detect if quick dial or the mini widget were involved.  The problem here, is that it only collected data on features already built, and generated no relevant insights for improving the product.

The next iteration became more controversial (hence the need for greater emphasis on permissions) as it started to report call log events that weren’t initiated via the Widget.  This made it possible to collect real world data on how often users were facing various use cases, revealing where the Widget work flows were weak and needed improvement.

I look forward to reporting my findings in full in a follow up post.

!! None of this is legal advice !!

Fascinating animation illustrating the nuclear weapons that have been detonated (thankfully nearly all for testing) on our planet between 1945-1998.  USA comes out badly with a crazy 1032 detonations (two used in anger), but surprisingly this only represents half of the 2053 weapons used to date.

Glad we’re still here.

Highly interesting lecture on how Fructose (a cheap fat/sugar replacment) is a poison and why we are feeding it to ourself (and our babies).

“Robert H. Lustig, MD, UCSF Professor of Pediatrics in the Division of Endocrinology, explores the damage caused by sugary foods. He argues that fructose (too much) and fiber (not enough) appear to be cornerstones of the obesity epidemic through their effects on insulin.” Youtube Link

I took away a lot from that 90 minutes, particularly that Ethanol is a carbohydrate (that’s why I’m getting fat), why you can’t burn calories with exercise, and Roberts lifestyle intervention for fat kids:

• Get rid of all sugared liquids - only water and milk.
• Eat your carbohydrate with fibre.
• Wait 20 minutes for second portions.
• Buy your screen time minute-for-minute with physical activity.

That last one sure sounds difficult!

Great chart on how much power is used by a Plasma TV (and other devices) while in standby mode (plus Video).

Despite all the pretty visualisations generated by the Google Analytics tool, essentially so far all it has been good for is recording how many people have been using my app.  This falls short of useful, now that I am at the stage where I want to redesign my interface to support an improved user workflow.

The next step is to come up with some kind of useful success vs. failure definition metric, which will allow me to analyse aggregated (opt-in) user behaviour and try to make design decisions accordingly.

Initially I thought this would be impossible for a “stateless” home screen Widget (with just two buttons) to collect data on wider user activity, until I realised that the widget regularly performs processing on the users call log data, meaning that outgoing calls could easily be matched against the most recent UI event, resulting in simplified (and anonymous) UI “Success” and “Fail” like events.

On my first pass, I am looking for these events:

Note to self: refine this list after further experimentation, and add extra scenarios to measure the success of changes made in version 2.

I am approaching the big two point nought milestone of the Last Caller Widget and feel the need to give it a cool new feature.

FEEDBACK
So far feedback from users has been mostly positive.  I have fixed a number of home screen and life cycle related issues, as well as responding to feedback from Sense UI users who felt the Widget needed more transparency to fit in with the HTC flavoured home screen.  For my next minor release I am only looking to clean up a few edge cases around withheld numbers and custom contact detail labels (both bugs) and to investigate one report of the Widget looking ugly on the Sony Ericson X10 (yet another custom home screen – still have to verify).

As of now, I am not able to reproduce any “Widget does not update” issues on the current Android Market build, on any test device (anyone still seeing this issue?).

Sequential Art - Anything I damn well please!

As this is free software done in my spare time, I don’t have to religiously chase after users with minority bugs (like I do in my day job) and can instead relax and start to think about the next cool feature.

NEW FEATURES
Looking through my email (feedback@zedray.co.uk), it seems people want a larger widget (with more calls listed), a smaller widget (4:1 just takes up too much of my valuable screen space), filtering by type (lots of people just want to see missed calls), open sourced (seriously, go to the Android Open Source project for more complete reference code), localisation (Russian & Chinese), and of course SMS support (complicates the UI in surprising ways hence I  am thinking of doing this as a paid app).

Of course you can only get so far by listening to your users, which brings me back to the origin story (blog post).  Basically (for vaguely work related reasons) I found myself analysing how well the native UI on the Nexus handles key uses cases (e.g. return the last call), and found some significant gaps.

My method came from some of Russels comments on measuring UI usability based on the number of clicks required to perform a particular task.  His argument centred on how important it is that we reapply this (very well established UI metric) to objectively analyse UI on touch screen devices (tablets in his case), as developers need to redesign their UI paradigms to support a completely new way of interacting with computers.

ANALYSIS
For these measurements I have chosen a very tight scope, covering only a few key use cases (wondering what more I should be adding?).  Also, all measurements are taken after the phone has been unlocked (not a valid assumption if someone adds this kind of functionality to the lock screen or comes up with a gesture based control, etc).

Note to self: repeat theses tests using trackball only.

As you can see above, the Last Caller Widget performs admirably in the first four use cases, making it by far the most useful Widget on my home screen (or for anyone who uses their phone as a phone).

Unfortunately that last use case is a dozy, as it turns out I have several numbers for the people I call most (go figure), hence I am always fiddling through the clumsy sequence in the native UI:

Call log -> Call info -> View contact -> Select number.

This is made slightly worse in 2.x as calls are sometimes grouped, adding yet another click to this already tedious chore.

This issue has been addressed under the contacts tab with the new 2.x “Quick actions”, which I’ll have to investigate as soon as I have finished working my way through the Google IO 2010 session videos.

Time for a statistics round up now that my “Last Call Widget” has breached the 10,000 downloads mark on the Android Market.

According to the developer console, the widget currently scores 4 out of 5 stars, taken from 83 mostly anonymous ratings.  Looking at the public comments (which are dated), it’s possible to see how several bug fixes and new features have increased the score as the product has improved over time.

Downloads
The Market reports raw numbers of 4,966 active installs for 10,814 downloads (i.e. 45%), which are good numbers for a free application downloaded off an App Store (I haven’t tried promoting it yet).  To get more in-depth information I had to abandan Flurry (due to its technical limitation) and adapt the Google Analytics plug-in to make it work with Android home screen widgets.

Unfortunately, Analytics’s web front end is designed around tracking website usage (supporting a variety of inappropriate use cases, i.e. session length), meaning I have to process the numbers somewhat before I get useful information.

Activations

The widget was enabled 23,390 times in its first 41 days.  By enabled, I mean that the user installed the widget on their Android home screen while the Analytics opt-out check box remained ticked (as it is by default).

To get a more useful impression of user loyalty, we need to understand how long the widget has been present on the users home screen.  As widgets are stateless (i.e. designed not to consume system resources), there is no way to accurately measure session length as you would do in a regular application.  So instead I opted to write a simple algorithm that generates a series of custom events over time, giving us an idea of how long each widget instance has been around.

Custom events make most sense when shown cumulatively.  In this chart, the number of “Enable” events takes off quickly, while the first update “0″, and the second update “12″ (i.e. twelve hours on screen) report lower numbers later on (as expected).  Looking at the chart, its obvious that there is a serious gap between users who enable the widget and then subsequently go on to use it.  This could be explained by users immediately disabling the widget (possibly a bug), deactivating reporting (optional in the UI) or simply by having data switched off on their phones (very common).

Looking at the subsequent events, there is a good conversion of users from 12 hours onward to 72 hours (I have omitted intermediate events for clarity), upto the highest value reported of 144 hours (i.e. 12 days).

As an added bonus, we also get to see how often each of the widget’s buttons were pressed.  Looking at the percentages, there appears to be an almost 1:1 relationship between button size and how oftern it was used.  With later users deciding to activate the “QuickDial” mode once it became available in later versions.

Three cool things about our weekend trip to Lisbon…

The Landing
The airport is so deep inside the city, that on landing the pilot may as well be driving a tour bus (if you look to your right you can see the Santuário do Cristo Rei, next is the Praça do Comérciothe headquarters of several Portuguese government ministries…).

Elétrico em Lisboa (Electric Trams)
Feeling like an old amusement part ride, the historic 28E (electric, not the regular 28 bus) will take you through winding streets, up and down hills and past the main tourist sites in the Lisbon old town.

Old Town Apartment
Enjoy getting lost as you walk along narrow winding streets, full of tiny shops and hidden family run restaurants.  Try and work out where the hills are on the map, and which set of steps take you where you want to go.  I suggest you either ask for directions of warm up that GPS!

“You did well on Twitter…” was the first thing I heard after taking absolutely no questions from a room packed with Android developers.

They had come to hear a real Android developer actually talk about Android development, as opposed to push some product or service (all the real developers were at yesterdays BarCamp).  I wasn’t trying to sell them anything, other than cram the benefit of several years of Android experience into the twenty minute attention span of the average conference attendee.

I had the slides (OpenOffice impress, not PowerPoint), I had source code (open on GitHub of course) and even an app on the market they could download while I demoed my framework on stage.

My plan was to keep it simple.  Concentrate on the hardest, and ironically most basic parts of the platform (i.e. keeping components with different life cycles held together), show where people often go wrong (service leakage, unnecessary RPC, etc), and top it off with a working demo for them to take away and learn from.

My greatest fear was for someone at the Google I/O (which I couldn’t attend) to have covered the same topic, or worse having recommended something contradictory (still waiting for Google to put those session videos up).  Additionally, a part of me kept wondering if someone was going to stand up and shout Balderdash (or some German equivalent) as I made my most bold claims and controversial recommendations.

But on the other hand, in the weeks I had to prepare the main slides (after finishing the framework code), I realised that I was going to be able to demo a bunch of things people normally don’t think about while developing.

Things like:

• How should your Service update your UI?
• When do you end your app?
• How do you make your app come back from the dead after your process has been killed?

I had turned up twenty minutes early to turn my development laptop onto a presentation machine (I knew the Dell/Windows XP/NIVIDA drivers would start fighting each other for control as soon as I plugged in the projector).  I set up my laptop screen orientation (no daylight glare), resolution (medium), projector (clone mode), brightness (full), Android emulator (correctly sized to fit the screen), mic (a nice wireless wraparound model provided by the technician) and tried out a cool little Logitech laser pointer/slide controller gizmo that (somewhat surprisingly) worked with OpenOffice as soon as I plugged the USB dongle into my machine.

It helped that I had tested my slides out on the same projector that same morning, and realised that my original colour scheme made the component diagrams illegible.  An issue easily solved during the days welcome keynote.

I checked the clock, and realised that I was ready with five minutes to spare (much better than trying to set up with everybody watching).  So I put the title slide up and sat down in the front row, looking over my equipment as the room began to fill.

I calmed my nerves by breathing slow and thinking through my main points, before someone politely asked me to vacate my seat.  Standing up, I turned around to see a packed room, with people shuffling around in the isles, and newcomers starting to chose their places on the floor.

Around now, it dawned on the organiser that the allocation of a “small” hundred seater room was going to be somewhat inadequate.  Instead of disrupting other parts of the conference, the moderator apologised to the room and opened up the double doors so people could watch from the corridor.

By my reckoning, there must have been north of a hundred and eighty people trying to watch in a room designed for half as many (I only found out later that some of my colleges had been turned away), which meant as I spoke I could taste the moisture in the air.

I kept calm, and walked the audience through my slides, skipped through some of the code examples  (20 minutes is not much time) and ran the framework demo with whatever time was left over.  The demo worked flawlessly, which I considered to be lucky as better men then me had seen their demo’s die that day (not needing a working internet connection always helps).

Surprisingly, when it was all finished no one put their hand up to ask a question.  Instead I was greeted by the sound of people quietly shuffling away to make changes to their apps.  Above all, developers don’t like to look stupid compromise their ego in front of their peers.

Presentation of “Architecture your Android Application” DroidCon 2010 are on Slide share, source code on GitHub, demo on the Market.

Oh, and that Twitter feed: