HTML5 = new world of hackable games

I believe HTML5 will create a new class of games: webbish games that, like the web itself, are hackable by design. These games will let you pull assets and data from across the web into your game world. And, they will let you remix, fork and share to your heart’s content. The result will be fun for people who like games — and huge potential for webmaking and learning.

The first glimpse I got of this was Jono Xia’s RunJumpBuild, a very simple side scroller level editor. While RunJumpBuild is rudimentary at this point, it shows an important idea: web games can pull anything with a URL into a game. A picture. A sound. A video. Data. RunJumpBuild is designed to let people create game levels that are made up of the web.

This may sound like no big deal. This is how millions of everyday web pages already work: displaying an image from Flickr; embedding a video from YouTube; pulling in Twitter and RSS data. But, just like movies, most games don’t work this way. Games tend black boxes, firewalled off from the social and creative assets of the web. IMHO, this is a missed opportunity for fun and creativity.

The other thing we’re starting to glimpse is games where ‘the web is the level editor‘. Of course, hackable games are nothing new. There is a long history of game modding. However, most modding uses level editors designed for a single game or game engine. HTML5 games will change this: the web programming stack (HTML, CSS, JavaScript) will become a universal level editor.

Enchant.js is an early example of this. It’s a rudimentary JavaScript framework for making 2D games. What’s notable is the invitation to ‘fork’ that is included in all the sample games. Press the ‘fork’ button and you get pushed into an editor that lets you work with the HTML, CSS and JS code that makes up the game.

While these developments are nascent at best, I believe they hold great potential for Mozilla’s webmaking work and for our efforts to promote HTML5 games through our app store. At the webmaking level, the potential is obvious:

• Games are where the people are
• Hackable games can teach people basic web mechanics
• -> e.g. a URL is how the web knows where things are
• They could also teach HTML / CSS / JS to non-programmers
• -> games = motivation to learn the ‘universal level editor’

With this in mind, we’re starting a number of HTML5 game experiments as part of our webmaking work. The most significant is Gladius, a game engine which uses the web’s most popular 3D game libraries. We’ll use Gladius to build sample games with a web native level editor as a way to test out the ‘hackable web games’ idea. We’re also considering hackable game experiments built around basic 2D game archetypes and learning-oriented game experiences using Processing.js. All of this is in very early stages of development.

Obviously, there are a bunch of big design questions that will come with this work. How do you use content from across the web to add fun and playability into a game? How do we create simple UX to onramp people who have never edited HTML / CSS / JS before? And how do you layer in the learning angle? The good news is we have many of the same questions with Hackasaurus and Popcorn, so we’ve got a head start.

As an example: we’ve already done alot of work with Popcorn to build an ecosystem of plug-ins that pull services and data into a web native video production. We have plugins that make it easy to throw Twitter, Flickr or GoogleMaps into or around your video. We should do the same with games.

Why does this matter? Because much of the HTML5 talk today treats games like a black box — using things like Google’s NaCl to port existing C++ games to the web. This is similar when we first started Popcorn, a time when the web was seen simply as a distribution channel for a black boxed video. Now we’re starting to show that video on the web can be about something totally new. We can do the same with games: to seed a whole new class of games that are actually designed from the ground up to work like the web, and that are more fun because of this.

Rumour has it that Dan Mosedale, Alan Kligman, Bobby Richter and Rob Hawkes are going to be blogging about some of our plans in this area in coming days. So watch for that. Also, if you know other examples of hackable HTML5 games please let us know about them. We want to work with any and all people who are exploring this space.

PS. Yes, there is a typo in the word ‘learning’ in the diagram above. That’s part of my charm.

Dan Sinker joins Mozilla as MoJo lead

I’m excited to announce that Dan Sinker will soon be joining us to lead the Knight Mozilla News Innovation Challenge (MoJo). MoJo is already kicking butt. It’s about to kick even more.

I knew Dan was the right guy for this gig when I learned that he a) taught web dev to journalists at Columbia College for 3 years and b) successfully ran Punk Planet magazine for 13 years. Web dev, journalism and gritty business chops to boot. Exactly the sort of things MoJo is about.

Dan’s other claims to fame include: a year as a Knight Fellow at Stanford; entertainingly imitating the Mayor of Chicago on Twitter; and belonging to a secret mailing list of menschy hipster dads with young children.

Of course, it’s what Dan is about to do with MoJo that has me most excited. When we first met over tap water in an AirBNB’d living room, we jumped right onto the topic of ‘wow we could build a huge and really awesome news innovation community‘. We’re still talking about this today. Dan wants to make it happen.

Dan is going to do an excellent job running the MoJo fellowship, for sure. I also believe he has the chops and vision to turn MoJo into something much bigger.

Welcome, Dan. See you on August 8th.

The challenge: reinvent ‘TV news’ online

Recently, we’ve seen a huge change in video online. The advent of web native <video> makes it possible to mash up moving images with social media, tie clips to data from across the web or, more simply, create simple transcript-based interfaces for navigating long pieces of video. Yet, despite the these capabilities, we’ve seen almost nothing in the way of new kinds of storytelling. Telling stories with video online today looks pretty much the same as it did when I used to shoot local TV news 20 years ago.

This is something we hope to change with the first Knight Mozilla news innovation challenge topic. We’re inviting hacks and hackers from around the world to answer the question: how can new web video tools transform news storytelling? People with the best ideas will get to bring them to life with a full year paid fellowship in a world leading newsroom.

The next ‘montage moment’

What do I mean by transform storytelling? Just that: taking today’s online video tools beyond the mechanical and obvious, bringing people, ideas and events to life in ways we haven’t seen before. To get your imagination going, think back to how visual storytelling emerged in the world of cinema.

The Lumiere brothers made some of the worlds first films. Workers going to a factory. A train arriving at a station. Etc. The Lumiere’s fixed frame wasn’t much to write home about in terms of story. But seeing moving photographs was hugely impressive to most people at the time. It was a technical wonder.

It took 25 years for Eisenstein to grab hold of this technical wonder and then say: wow, I bet you we could tell a more powerful story if we varied the shots a bit and then edited them together. With Potemkin, he invented the visual language we still use to tell stories today: montage.

The fundamental technology didn’t change in those 25 years. The Lumiere’s knew how to splice film and move the camera around. Eisenstein’s breakthrough was to use basic film technology to tell a story in a new and creative way. Which is very much like where we are at with web native video today: huge technological potential just waiting to be seized for creative storytelling. What we need now is a ‘montage moment’ for the web era.

Open video: a huge palette of awesomeness

The potential of web native <video> truly is awesome: we can now link any frame within any video to any other part of the web. This was hard to do in the world of Flash video. The introduction of the HTML5 <video> tag over the last two years has made it easy.

Early experiments and demos hint at the potential of this new open video palette. With the recent State of the Union, PBS used Mozilla’s popcorn.js tools to synchronize their live blogging with the timecode of the President’s speech:

The same tools have been used to show how transcripts can be used to search and then navigate immediately to anywhere within a long clip. This demo from Danish public radio shows how this can work with web native <audio>. The same thing could easily be done with video.

Of course, the big potential is in connecting video to the massive amount media and data that already exists all across the web. Imagine if you could weave the sum of all human knowledge seamlessly into your news story or documentary. That’s now possible. This book report demo shows the basics concept, with a student connecting her narration to wikipedia articles and news reports.

Google and Arcade Fire took this idea a step further, pulling moving images from street view and Google Earth into a rock video. If you enter your zip code, your neighborhood becomes a character in the narrative in real time.

The Japanese based Sour-Mirror went even further, pulling you into the video. Enter your Facebook ID and turn on your camera, and then you become a character in the band’s video. Again, in real time.

These demos make an important point: the line between what’s in the frame and what’s on the web is dissolving. Or, put nerdily, timecode and hypertext are fusing together. They are becoming one.

Are you the next Eisenstein?

Despite all the niftyness, there is a problem: these demos do not yet tap the open video palette to tell stories in meaningfully new ways. Open video tools like Mozilla’s Popcorn and Butter provide a starting point. But they need people with a creative flair for both web technology and storytelling to bring them life. Which is exactly why Knight and Mozilla threw out ‘how can new web video tools transform news storytelling?’ as our first MoJo challenge question.

We’re hoping that you — or someone you know — is up to this challenge. If you think you are, you should enter the MoJo innovation challenge. All you need to do is: draw up a napkin sketch showing how you might tell a story in a new way with open video, write a brief paragraph about it and then submit it online. If your idea is solid, you’ve got a good chance at a fellowship where you could actually bring it to life at the Al Jazeera, BBC, the Guardian, Die Zeit or the Boston Globe. Who knows, maybe you could be the Eisenstein of open video?

Find out more about Knight Mozilla News Innovation Partnership on the MoJo web site. Or enter the MoJo news innovation challenge today.

MoJo Interview: popcorn, video and news

I asked Mozilla Web Made Movies Lead Brett Gaylor excites him about news and video on the web today. He said: “Open video can help re-establish journalism as the fifth estate.” As the first Knight Mozilla news challenge topic is about video, I asked Brett to say more about this:

Brett riffs on the idea of a ‘wonk-a-pedia’ that uses open video on the web to connect video clips of politicians to their voting records, past positions and so on.

The current MoJo challenge topic is about exactly this question: how can new online video tools transform news storytelling? Tying video snippets to other web content and data. Letting audiences join into the debate. And do on.

If you’re a talented web developer excited about HTML5 video, you should enter this first MoJo news innovation challenge. If you want advice or inspiration, head on over to popcornjs.org or Mozilla’s popcorn IRC channel. Lots of people there who can help you out.

Just think of it: you might end up at BBC, the Guardian or one of our other news partners working on the next big thing in online video. All you have to do to get started is submit your idea.

—

This post is part of series of interviews with people involved in the Knight Mozilla News Innovation Partnership (MoJo). Find out more about the partnership on the MoJo web site or enter the MoJo news innovation challenge today. I’ve also posted a really simple primer on MoJo.

MoJo interview: Al Jazeera + innovation

I asked Al Jazeera’s Mohamed Nanabhay what excites him about news innovation on the web today. He said: “I’m excited by the sheer velocity of change going on. The web has taken the concept of live news and stretched it to the limit”. As a part of our joint work on the MoJo, I asked Mohamed to say more about this:

Mohammed also talks about the merging of traditional media and social media: a phenomena that has shaped a great deal of the coverage of recent events in the Middle East.

As Mozilla Knight News Innovation partner, Al Jazeera English will be hosting a fellow who will work on web apps that explore this trend. I know that Mohamed is especially interested in fellows who want to work on grassroots reporting, social media and video, which  ties into our current MoJo challenge topic.

If you’re a talented web developer or designer, you should enter the MoJo news innovation challenge. Who knows: you might end up at Al Jazeera building the next big web news app.

—

This post is part of series of interviews with people involved in the Knight Mozilla News Innovation Partnership (MoJo). Find out more about the partnership on the MoJo web site or enter the MoJo news innovation challenge today. I’ve also posted a really simple primer on MoJo.

MoJo news challenge: what is it?

Earlier this week, Mozilla and Knight launched our first news innovation challenge (aka MoJo). We’ve had good buzz, but also people saying ‘what is this thing?‘

I wanted to come up with the simplest possible explanation. At the top level, the idea behind MoJo is:

Back five individuals to build mind blowing web apps
in collaboration with world-leading news sites.

Broken down a little more, Mozilla and Knight plan to:

1. Find five incredibly talented web developers and designers.
2. Give them a paid fellowship in a world-leading newsroom for a year.
3. Using these newsrooms as a lab: build open tech and web apps that shape the whole field of news.

If you are an incredibly talented web developer and designer, you should enter the challenge over the next five weeks. Here is how:

a. Submit your idea for a news web app or tool. All you need is a napkin sketch to start.
b. If your idea is good, the brightest minds in web tech and news will help you build a prototype.
c. If your idea is really good you’ll have a chance to build out your app during a one year fellowship.

Why does Mozilla care about the future of news on the web? Because:

We want to see the open technology and participatory culture
of the web become a core part of how news is made. This is
good for the web and good for society.

If you want a slightly more detailed but still simple overview of MoJo, you might want to watch this two minute explainer video:

Phillip Smith’s 10+ reasons to enter the Knight-Mozilla news innovation challenge is also a great read. And, of course, you should also check out the MoJo web site and the first challenge around open video and story telling.

Media, freedom + the web: berlin talk

As I pointed out a while back, this year is Marshall McLuhan’s 100th birthday. It’s a good time to be thinking about media and the web: in particular about how the free and open medium of the web is shaping all media that came before. Increasingly, this is a theme for Mozilla Drumbeat in 2011.

Why now? Yes, partly because it’s Marshall’s birthday. But more importantly, we’re at a key juncture: traditional media are increasingly reinventing themselves by tapping into the essence of the web; at the same time monopolies in spaces like social networking and mobile apps are calling the freedom of the web into question. Things could go either way: open or closed.

Back in February, I explored this theme in the annual Marshall McLuhan Lecture at the Canadian Embassy in Berlin. I’ve re-recorded the talk and posted it here:

It’s a long talk: 40 minutes. If you just want to get a feel, you can flip through the PDF version. Or, you can download an mp3 audio version or an ogg video version.

At a high level, I believe we have to make a number of critical choices in coming years that will impact media and society for decades to come. My three top level points are:

We rarely call it out, but the same basic principles that make free software and open source great are also baked into the very fabric of the web itself. The web gives us the freedom to use, study, remix and share — that’s what we are all doing at a massive scale. We do these things because they are baked into both the technical building blocks and the culture of the web. When we think about the web as the medium that is shaping our times, it’s important to remember that this kind of freedom that is central to what’s going on.

McLuhan said: “The next medium, whatever it is – it may be the extension of consciousness – will include television as its content, not as its environment, and will transform television into an art form.”

This has happened. And it hasn’t just happened to television. All media have become the content of the web. As a result, all media are wrapped in this context of freedom: in a world that lets you bend and share without asking permission. The initial reaction from old media was push back. But times are changing. We’re very clearly entering a phase where smart media players are using the essence of the web to reinvent themselves. Eg. witness the Guardian and Wikileaks or Al Jazeera in Egypt.

The context is a web built on freedom. The opportunity is that all media are reinventing themselves in this context. If we seize this opportunity, we can bake things like transparency, remix and sharing into the media culture and practice for the next 100 years. That’s what we’re trying to do with Mozilla Drumbeat projects like popcorn.js: build tools that give filmmakers and journalists access to the essence of the web. If we succeed, we also bake the web into how whole industries work and think.

Of course, there is another direction we can choose: we could close down the web. Tim Wu talks eloquently about this in his book the Master Switch. Talking about media empires in the last 100 years, he says: “Open eras tend to last for about 15 – 20 years. And then they flip into being more closed. 
We may be at the beginning of the closing with the internet.”

Specifically: we could give up our privacy and identity to one or two social networks; we let one or two companies decide who gets to innovate and create software; we could let governments decide whether we get to access the internet at all. The result would be a very different web than the one we have now.

It’s this point about choice that makes media such an important theme for Mozilla and Drumbeat in 2011: now is the time to aggressively, creatively and playfully promote web technology and web thinking in the broader world of media. What we do now will shape media — and society — for a long time to come.

I’d love to get people’s feedback on the ideas in this talk. And, even more, I’d love to see people building things and playing with the theme of media, freedom and the web as part of Mozilla Drumbeat in 2011.

This is the third in a series of posts about media, freedom and the web. I’m hoping to do more, including a few posts on the future of cinema.

Definitions: media, freedom, web.

I had a good conversation with John Udell the other day (thanks!). We were making a list of concepts people need to internalize if they want to think like the web. The next morning, a particular concept jumped to mind: Openendedness. The web is never finished. It’s built to be built on.

The idea of openendedness is a striking one when comparing the web to other media. Newspapers. Films. Books. Television. Comics. Long playing records. Every important medium since the printing press has organized itself around discrete objects that have a start and a finish. The web does not.

As I posted last week, I hope to spend a good chunk of 2011 exploring the openended nature of the web and the way it’s shaping other media. Riffing on last year’s Mozilla Drumbeat theme, I’m using the phrase media, freedom and the web as an umbrella to throw ideas and projects under. Given this, I figured it would be useful to unpack what these words mean to me. Let’s start with …

Freedom

The kind of freedom I am talking about: The openendedness of digital things. Bulding things we can build on.

Loosely, I’m talking about software freedom here. In 1986, Richard Stallman declared that software should have four freedoms. Paraphrased they are: the freedom to use, study, remix and share. While many people don’t buy into the four freedoms per se, the basic ideas are widely accepted. Some rough approximation of use / study / remix / share is what most people mean when they say ‘free’ or ‘open’ in relation to technology.

Freedom in this sense is useful conceptual frame that not only helps us understand the web but may also give us tools to reinvent the media of the past. It’s use / study / remix / share that make the web openended. The same frame offers a useful set of design tools as we start to reinvent media more widely.

Web.

My simple web definition: An infinite box of LEGO that lets anyone build (almost) anything.

Consciously and unconsciously, we have baked freedom and openendedness into the very fabric of the web. At the most literal level: huge parts of our internet culture and economy rest upon a foundation of free software (Google, Amazon, etc.). But the essence of these ideas reaches even further: web wouldn’t be what it is today without the ability to look under the hood, get your hands dirty, and fix what doesn’t work.

I find best way to explain this to most people is to talk about the LEGO that the web is built with. HTTP. HTML. CSS. JavaScript. Etc. At its core, the web is these building blocks. The wonderful, diverse (and occasionaly horrible) web that now spans the world is made up primarily of these things.

Importantly: we have this web because so many of us were able to use this LEGO to invent, build and bend things without asking permission from others. No one has to ask Tim Berners Lee less if they want to use HTML or Brendan Eich if they want to use Javascript. No one had to ask to view source. This is the very practical magic of the web.

Openendedness is central here. The network and the LEGO blocks are built to be built upon. They are built this way not only technically, but also culturally and conceptually. People steeped in the web — and many people who have grown up with it — don’t think about finishedness in the same way we their parents did. Iteration. Adaptation. Extension. These are the things they value, and that might benefit all media.

Media.

My simple definition for media: The things that connect us. The stories we tell. The people who tell them.

We all talk about media. But we talk about it differently. Some of us talk in the abstract sense that McLuhan meant: the material essence of a particular medium. Print. Film. Web. Others talk more tangibly about the stories we tell and how we tell them. Books. Movies. Social networks. Still others personify ‘the media’, talking about the people who make it. ‘The media is screwing up politics and democracy’ or ‘the media sucks’.

As the web era moves from disruption to reinvention, it’s important to contemplate what is happening with all of this. Things are changing at all levels. The material essence of what we used to call print or film. The ways we tell stories as books and movies. And, most dramatically, the people who make media (which we increasingly recognize to be ‘all of us’). All of these things are in play.

For me, the most interesting aspect of this reinvention is this question of finishedness vs. openendedness. Typesetting and the printed page were rigid and closed. The wiki and the web page are openended. We use them for seemingly similar purposes, but to very different effect.

As we reinvent and make choices, we have a chance to build the openended nature of the web into all media.

—

Why hop through these three concepts? Partly to map the terrain for a series of experiments. I’m hoping Mozilla Drumbeat can explore media, freedom and the web in some very practical ways during 2011.

However, McLuhan also urged us to ‘contemplate what is happening’ with media if we want to shape it. This requires a shared vocabulary, a broadly understood set of concepts. For me, the openended nature of the web is one of these concepts. Many more people need to understand that the web is built on freedom, even if we end up using different words.

In my next post, I want to do an old media / new media case study. I started out my career as a filmmaker, so it’s probably about how cinema might be reinvented on the web.

This is the first in a series of posts about media, freedom and the web.

McLuhan, freedom and the web

If he were still with us, Marshall McLuhan would turn  100 this year. What to get him for his birthday? I’d get him LEGO. Partly because LEGO’s fun. But mostly because it’s a great metaphor for open-ended nature of the web. As a medium, the web has one big message: build me.

I plan to spend 2011 exploring this message: looking at the open-ended nature of the web and at how it is shaping other media. I want to play with this very practically: making software, web sites, events, etc. that fuse old media with the culture and technology of the web. Riffing on last year, ‘media, freedom and the web’ may well be a major Mozilla Drumbeat theme for 2011.

Why now? Partly it’s poetry. It’s McLuhan’s big birthday. Partly it’s opportunistic. Mozilla Drumbeat is experimenting with cinema, journalism, language and the web. But the biggest reason: this feels like an important moment in media, one where we move from the web as disruption to the web as reinvention.

McLuhan famously said: We shape our tools, and then our tools shape us. This is very important right now.

We invent and play with media, and then things slow down. The printing press. Pamphleting. Newspapers. And then things slowed down. Motion pictures. Splicing. Montage. And then things slowed down. In all of this is a process of disruption and reinvention. Breaking down what was, shaping what will be.

With the web, we’re still sitting on the edge between disruption and reinvention. For example: Hollywood and the TV networks have been disrupted by camcorders, youtube and a legion of 11 year olds. But we have yet to see the new forms of web cinema that are clearly around the corner. We haven’t yet met the next Eisenstein or Vertov. We haven’t had our montage moment. These intersections between old and new media are what matter most when we think about shaping our tools. We get to invent and reinvent. We get to make choices.

Many have credited McLuhan with predicting the web 30 years before it was invented. Whether you think it was a prediction or not, what he wrote in 1962 helps us think about the intersection we’re currently staring at:

The next medium, whatever it is – it may be the extension of consciousness – will include television as its content, not as its environment, and will transform television into an art form. A computer as a research and communication instrument could enhance retrieval, obsolesce mass library organization, retrieve the individual’s encyclopedic function and flip into a private line to speedily tailored data of a saleable kind.

It’s clear that the web has embraced and engulfed almost all media that came before. Print. Photography. Radio. Film. Television. Or, from another angle > Storytelling. Prose. Journalism. Cinema. Conversation. As we all have witnessed, these media are all caught up in a wave of disruption sparked by the web.

Which brings us back to freedom. With every new medium there are always waves of freedom and openness, followed by something much more closed. What’s different with the web is that we’ve built it around some very particular notions of freedom and openness from the start. We’ve cast it in a forge of transparency, sharing, remix and participation. We’ve baked these things into, sometimes by accident and sometimes on purpose. We’ve built the web as an open-ended system. It’s built to be built on.

It’s important to remember this as we move from disruption to reinvention. The choices we make in the next few years will determine whether this freedom and open-endedness remain at the heart of the web. Our choices will also shape what everything from journalism to cinema to language become: our choices will determine how these media are reinvented.

We could choose a web that keeps us connected but drops the freedom. Imagine a web where we trade privacy for convenience. Where your digital identity and credentials are locked up by one company. Where a handful of players get to choose which software applications make it into ‘the market’. Tim Wu’s recent book warns of this future, reminding us that past media disruptions have started open then fallen into a pattern of rigidity and empire.

Or we could make conscious choices that keep freedom and openness baked into the web. We could ensure that the next pieces of the web — video, mobile, identity, social, etc. — are built on the same open principles as TCP/IP, HTML and JavaScript. We could fuse the culture of creativity, innovation and open-endedness into the broader world of media as it reinvents itself. This is what Mozilla and so many others are working to accomplish. And, given the head start that we’ve got, it’s possible.

McLuhan provides a helpful reminder here: paying attention and making good choices can shape the future. He wrote:

… there is absolutely no inevitability as long as there is a willingness to contemplate what is happening …

What is happening now is that we’re at a crossroads: we’re making choices about the web today that will shape society and media for a very long time to come. Choices about technology. About whose in control. About freedom.

It’s this set of choices that makes me want to dig into the connection between media, freedom and the web right now. Over the coming weeks, I’ll post on different aspects of this. My next post will be on what the words media, freedom and web mean to me — defining the terrain a little better. I’ll then jump to some of the practical things we can do to play with these ideas, especially as they relate Mozilla Drumbeat work on cinema, journalism and language.

This is the first in a series of posts about media, freedom and the web.