One of the greatest benefits the data-driven development paradigm affords a free-to-play mobile studio is the ability for users to communicate their preferences through actions, rather than words: analytics serves as a conduit through which feedback is transmitted between the player base and the developer, allowing games to be improved upon and optimized for user taste, not developer intuition or legacy insight.

But game development is an artistic endeavor, and some producers – who are artisans, by any reasonable standard – bristle at the thought of algorithmic, hyper-incremental game design. While game development has become increasingly informed by analytics, very few studios instrument the subjective aspects of design: aesthetic consistency, thematic appeal, game controls, etc. Terms like “juiciness” continue to pervade the industry.

As well they should. But just as no free-to-play game should be developed without input from a human, no free-to-play game should be developed in an intuition bubble. Freemium product development is agile, iterative, and data-driven by nature; what follows are three tactics for effectively introducing elements of data-driven development process without sacrificing creative discretion.

Define done

One of the concepts at the core of agile software development is the “definition of done”: product feature specs must be produced in sufficient detail to allow the team to determine when the development process can be concluded.

Defining done for software that exists to serve a discrete, objective purpose – say, translating a page of text from one language to another – is, by virtue of being measurable, easy to do. Defining done for an art asset, or a game design document, or a game economy model, is much more difficult: subjective value varies from person to person, and the state of progress of fundamentally immeasurable artistic undertakings is fluid, not binary (“done” or “not done”).

Defining done is important, but the process of defining done is so difficult — and the repercussions of not enforcing it are so dramatic — that many teams avoid ever doing it. The truth is that, no matter how well a team of people work together, at some point decision-by-committee becomes impractical; defining done must sometimes fall on the shoulders of the producer as a unilateral determination.

When this is undertaken at the beginning of a well-defined sprint, based on thorough product specs that have been communicated clearly, then conflict is easy to avoid. But when a unilateral decision is made as a sprint is drawing to a close, based on nebulous, imperious intuition, not only is conflict unavoidable but so too is development crunch.

Avoid not knowing

Intuition can be sticky; human beings are capable of clinging to preconceived notions for far longer than those notions hold true (if they ever did). Part of the liability introduced into free-to-play games development by producers with legacy experience in shipping console and desktop games is that user preferences can be subverted by an intractable and wholly anachronistic worldview.

The unknown can only be elucidated by data, and the more recent that data, the better if reflects the realities of a rapidly evolving marketplace. A/B tests are not a panacea, but they are essential in free-to-play game development – not only to shed light on preferences for entirely new mechanics but to reify previously-tested design decisions.

Which begs the question: can a producer that isn’t familiar at even the most basic level with common analytics frameworks and data-driven design techniques be effective in a free-to-play environment? Free-to-play games can be improved upon in-market using a panoply of metrics, but doing so requires a skillset that wasn’t previously relevant: a basic understanding of quantitative methods and the technology infrastructure required to test.

Promote transparency

Iterative development is undertaken because customer needs are never static: they change and evolve over time, and products that don’t adapt as customers do are left for dead in the marketplace.

A second core component of agile software development is the stand-up meeting: the entire team congregates in the same room and announces what they accomplished yesterday, what they plan to accomplish today, and whatever obstacles they face in completing their work. Complementing the daily stand-up meeting is the burndown chart, where the tasks allocated to the sprint are graphically arranged in such a way so as to visualize the work remaining.

An additional element of transparency that benefits the free-to-play development process is the metrics billboard: a highly visible, centrally located television in the development area that cycles through the game’s relevant metrics post launch. The metrics billboard not only prevents team members from becoming ignorant of the game’s performance, but it underscores the studio’s dedication to data-driven development.

This transparency is important: it helps team members understand development bottlenecks and prevents crunch. But perhaps most importantly, transparency provides the team with the opportunity to collaborate and quickly unify its efforts in response to data.

By promoting transparency, the team allows development shortcomings to be addressed immediately and convenient yet ultimately ineffective design strategies to be dispelled on the basis of objective analysis.

As a strategy

Defining done, avoiding ignorance through testing and measurement, and removing opacity from the development process contribute to an environment in which data reinforces creative insight without supplanting it. Data is the lifeblood of the freemium model, but it’s not prescriptive – the way in which data is used to improve upon the product and bring delight to consumers is an exercise which is enhanced by experience and insight.

Eric Seufert is a quantitative marketer with a specific interest in predictive analytics and quantitative methods for freemium product development. He blogs frequently at ufert.se (link: ufert.se), and his forthcoming book, Freemium Economics (link: freemiumeconomics.com), will be published by Morgan Kaufmann in early 2014.