Web browser – a playground for creative minds

Written by Matej Fandl on September 28, 2017

What do modern browsers offer to people who like to experiment and build things?

When it comes to doing novel or unusual things with computers, one needs to think outside of the usual patterns and combine different media. One needs an environment that allows for both quickly writing prototypes and building apps that people can use without too much hassle. There are beautiful tools that help people working at the intersection of art, design, science and technology make their ideas alive and move the world of computing forward. There is openFrameworks, Processing, Cinder, vvvv and many more. These make things like drawing on the screen, playing sounds, rendering text, network communication and interfacing with input devices like cameras, microphones and 2.5D sensors easy. Writing a software that can do this yourself is usually hard even on a single platform, not to mention if you need to target more than one.

The almighty?

While the aforementioned tools were developed by purpose for so called creative coding, over the years, there grew another platform with similar capabilities, but different focus. A platform, that is widely adopted and that thanks to active community and standards allows developers to do amazing things. A web browser. But what’s the big deal here? Building web pages? Yes, but …

You know how a typical webpage looks like. There is some text you can read, some images you can see and some links that allow you to navigate to other webpages. And then there are, in my opinion, more interesting things you can do. While (some of) the tools mentioned above allow exporting projects for web, I argue that it makes sense writing your software directly in web technologies and for the web.

This article is aimed at programmers who would like to get an overview of what is possible nowadays in the web browsers and would like to start programming interactive web applications. Professional web developers will likely learn nothing new. Of course, these informations won’t be exhaustive. I will mention a couple of things I find interesting. I will also add examples of projects that are worth knowing and are not regular webpages. Anyone curious enough can find a lot of information on the topic without trying too hard.

After the Flood – this needs WebGL2, which is experimental.

First things first

This is an article by a programmer for other fellow programmers. There will be a lot of weird words for anyone who is just starting learning to code. Please do not let this demotivate you. The primary target groups of this text are

Important thing is – I won’t mention any libraries here. While there is great benefit in using them, libraries are beyond the scope of this article and may involve baked-in abstractions that might shape the outcome of your work a bit too much. It is a bit harder to use the platform API’s directly. But by learning what’s there and how to use it, you get more freedom in architecting your programs and will be able to decide better on which libraries and frameworks to use. I will first mention the language(s) you can use to program browser applications and then some of the superpowers of modern browsers. Not all the things I mention work in all the browsers (yet). Some API’s are subject to change. I am writing this to convince those of you, who have a lot of ideas and who do not program for web browsers, to give it a try. The more minds here, the better.

Languages

Web pages are programmed in HTML, CSS and Javascript. This article will not focus on things related to HTML and CSS. If you want to learn more about these technologies, click the links.

Javascript is the language of the web, the only scripting language browsers understand. There are many languages that compile to Javascript, like PureScript, Scala.js, BuckleScript, TypeScript and others. This is good news for you if you are already programming in something else. You might be able to hop on the web programming train faster. Still, however, becoming proficient in Javascript is an advantage. It helps you understand the ecosystem much better. Especially if you want to write programs that are computationally intensive and need to run in real time and smoothly.

There is a very positive thing happening now – we are getting WebAssembly, web’s own assembly language. Wasm will allow you to compile C and C++ (and more in the future) programs for the web. This will result in both better performance you can squeeze from the platform and a library ecosystem boost.

Drawing

One of the things I mentioned in the beginning of the article is drawing in both 2D and 3D. There are standard HTML elements which you can use, together with CSS, to build rich visual presentations. Apart from placing images, rendering text, colouring quads, you might also want to do advanced 2D and 3D drawing. HTML5 brought us the canvas element. Canvas is a rectangle you can use to draw when manipulating individual pixels is necessary. It comes with two context – the 2D rendering context for 2D drawing and WebGL context for 3D.

2D

The 2D rendering context allows for variety of common operations. From drawing and colouring rectangles, images, paths, rotating things etc, to accessing and modifying values of each individual pixel . Many web games are rendered using the canvas element’s 2D context (slither.io is one of the examples). The advantage here is, that it runs almost everywhere and for simple things, where you do not need to write thousands of elements 60 times per second, it just work flawlessly. Access to pixels allows you to do things like computer vision, photographic filters or custom drawing tools directly on the frontend.

3D

And for the more complex things, there is WebGL. With WebGL, you can draw both 2D and 3D. You can draw thousands of things at once in realtime, while achieving good performance even on mediocre devices. You can render both realistic and stylized graphics with lights, post-processing effects like bloom or DOF, shadows etc. Apart from rendering, you can use the WebGL to run simulations, or just compute things directly on the graphics card – thanks to tiny, but extremely powerful programs called shaders (have a look at The Book of Shaders).

Currently, there is WebGL2 being worked on, which makes the 3D rendering capabilities of browsers even more powerful. Still marked as experimental, but already usable in Firefox and Chrome.

VR

Keep your eyes on WebVR, you can start developing immersive VR experiences right away.

Vector graphics

SVG is a standard supported by the browsers. This means, you can just draw vector graphics when it makes sense to you. You can animate it with CSS and Javascript and you can easily make it interactive since it supports the traditional HTML element events such as click and hover. By working SVG, you can generate visuals that look good both printed and on screen. With canvas, this is also possible, by saving what is drawn as image. But the larger the print, the larger the canvas needs to be. SVG saves you from this problem – just choose wisely based on your needs.

Working with sound

If you simply want to play/pause a sound file, you can do this with the HTML Audio Element. Should you need more control, there is the Web Audio API. This is a tool for sound engineers. You can use it to generate music or to modify sounds with various effects. You can even analyse sound with the built-in Fast Fourier Transformation and use the result as input for other parts of your program.

Access to input devices

There obviously is easy access to standard input devices such as mouse and keyboard. You can read the mouse position, listen to events such as clicks, drags and keypresses. Gamepad API allows you to read input from gamepads. With the MediaDevices interface you can also get access to microphone, camera and do screen sharing. Want to build a musical instrument that generates music based on the users facial expression? Or a video game controlled with voice? This makes it possible.

Location data

A powerful feature of the web browsers is the ability to query user’s location with the Geolocation API. Here, the possibilities are endless. From Pokemon GO like web games to pages that allow you to save notes on specific places or communicate somehow with people around you.

Resources on the internet

This is obvious, but still worth noting. It is very easy to make HTTP requests from the web browser and many public API’s and sources of data. These mostly come with a standard JSON format, easy to use with Javascript. This is a big advantage, since you can experiment very fast and make things that react to what is happening outside. Like interactive stories where what is happening is influenced by what is being discussed on Twitter, or based on what is the current weather in the readers location, or…

Being offline

While most of the people use a web browser to browse the internet and are therefore online, you can build web apps that behave like native apps. People can save them to the home screen on their mobile phones and even use them offline. If you want to learn more about this, read about the Service Workers and Progressive Web Apps.

Conclusion

By listing some of the features of modern web browsers above, I was trying to show, that this is a platform worth considering. It is powerful now and constantly worked on by many brilliant minds (who you can join by becoming a member of various W3C Community Groups). Most of the applications you can think of can be developed directly for the web browsers. They will run fast and people will be able to use them without downloading anything. Each update will directly influence everyone, no need to support archaic versions of you software.

If I forgot to mention your favourite feature, or if you have any feedback, please let me know in the comments. I will happily update the article.


Mladý pes, 2015, all rights reserved ;)
Privacy policy