Let’s make a language – Part 1b: Isian Phonology

This will be a much shorter post than the one last week, since we have all the theory bits out of the way. This time, we’re solely focusing on the sound system of our “simpler” conlang, Isian. Rather than just give a list of sounds, though, I’ll try to justify some of my choices as we go.

Isian Consonants

Labial Alveolar Palatal Velar Glottal
Nasal m n
Stop p b t d k g
Affricate tʃ dʒ
Fricative f s ʃ x h
Approximant w l r j

Isian has a total of 19 consonant phonemes. None of them are too exotic, though monolingual American speakers might have a little trouble with /x/, the “ch” sound in German acht or Scottish loch. Everything else should be familiar. If you don’t know the IPA symbols for the palatal consonants, that’s okay. In order, /tʃ dʒ ʃ j/ are the initial sounds of church, judge, shut, and yet. Also, the /r/ phoneme can be either a tap [ɾ] like Spanish or an approximant [ɹ] like English, though the first pronunciation will be the “official” one.

So why these particular 19 sounds? Well, Isian is supposed to be easy to pronounce, but I still want it to look and sound a little “foreign”. /x/ accomplishes this feat (for Americans, anyway).

English speakers might notice what’s been left out. There’s no /v/ (as in view), /z/ (as in zip) or /ŋ/ (as in *sing). That’s all right, because of allophones. Between vowels, /f s ʃ/ can sound like [v z ʒ] (the last as in French jour or English azure), and /x/ can disappear altogether, instead making the vowel before it sound a little longer. Or it could sound like [h], if the two vowels it’s between are the same. So we might have /taxa/ pronounced more like [taha], but /tixa/ as [tiːa]. Some of our fictitious speakers might instead substitute the voiced velar fricative [ɣ]; we’ll say that this is an older and more formal pronunciation.

In the same way, /m/ and /n/ will assimilate to a following consonant, except approximants and /h/. Before a labial, /n/ becomes [m]. Likewise, /m/ comes out as [n] before an alveolar. Both of them will subtly change to [ɲ] before palatals and [ŋ] before velars.

There are no TH sounds, since those are relatively rare, and Isian is meant to be fairly average. For the same reason, we don’t have any phonemic alterations like palatalization or aspiration going on. Voiceless stops might sound aspirated at the beginning of a word, like English, or not, but this can be explained away as a dialect feature.

Isian Vowels

Front Central Back
High i u
Central e o
Low a

Isian’s vowel system is an average one, with the five cardinal vowels. But we’ll embellish it a little with some allophonic alteration.

First, these aren’t the only vowel sounds possible. We’ll say that any of the three “lower” vowels /a e o/, when followed by a /j/ or /w/ consonant, creates a diphthong, a kind of combination of two vowels in the same syllable. It doesn’t take much math to see that this creates six diphthongs: /aj ej oj aw ew ow/.

  • /aj/ is about the same as the English long-I sound in lie,
  • /ej/ is close to the English long-A sound in lay,
  • /oj/ is pronounced like in English toy,
  • /aw/ can be the sound in English law or loud (we can write this off as dialect differences),
  • /ow/ is the English long-O in low,
  • /ew/ isn’t in English, but it’s the first vowel sound in Spanish or Italian neutro. We’ll say that some dialects pronounce it as [iʊ], like English few.

So, even though we have only five vowel phonemes, thanks to diphthongs, it seems like we have 11.

Second, we’ll say that a few vowels change a little before certain consonants. /a/ becomes [æ] (English ash) before the palatal consonants /tʃ dʒ ʃ/. And we saw above how vowels before /x/ might become lengthened. Finally, although we haven’t discussed syllables and stress, we’ll say that unstressed vowels tend to be “reduced” in fast or colloquial speech. For example, an unstressed /a/ might sound like a schwa ([ə]), like in English about.

Orthography

Orthography is, basically, how a language is written. Isian certainly isn’t going to have its own writing system; we’ll just use the alphabet. But we need a way to convert the phonemes into letters. English, of course, is notorious for being hard to spell, but Isian has far fewer phonemes, so it should be easier to fit into 26 letters.

Most of the phonemes can just be written as the appropriate letters. That works just fine for all the vowels, as well as the consonants /p b m f w n t d s l r g h/. The remaining six sounds need a little more thought. Here’s what we’ll do:

  • /k/ will usually be written as c, but k when it comes before /i/ or /e/. (This is mostly an aesthetic change. There’s nothing stopping us from writing k everywhere.)
  • /tʃ/ will be written ch, like it is in English. The same for /dʒ/ as j, /ʃ/ as sh, and /j/ as y.
  • /x/ can be written as kh. We can’t use ch, like German, since it’s already taken, and x would give English readers the wrong impression. Sometimes, you have to compromise.

So our full orthography for Isian looks like this:

Written Phoneme Description
a /a/ a in father; a in cash before ch, sh, and j
b /b/ b in boy
c /k/ c in cat; only used before a, o, or u
ch /tʃ/ ch in church
d /d/ d in dog
e /e/ e in Spanish peso
f /f/ f in fish
g /g/ g in go (always a “hard” G)
h /h/ h in hard
i /i/ i in French fini
j /j/ j in jet
k /k/ k in key; only used before i and e
kh /x/ ch in German nacht
l /l/ l in list
m /m/ m in man
n /n/ n in note
o /o/ au in French haut
p /p/ p in pit or top
r /r/ r in run or Spanish cero
s /s/ s in sat
sh /ʃ/ sh in sharp
t /t/ t in top or hot
u /u/ ou in French sous
w /w/ w in wet; creates diphthongs after a, e, or o
y /j/ y in yes; creates diphthongs after a, e, or o

The next post will switch over to Ardari. When we come back to Isian, we’ll make these sounds into syllables, then into words.

Repeatable Random Numbers with xorshift+ (JS)

Like most languages, JavaScript has a random number generator: Math.random(). It’s not the best, but it’s good enough for many purposes, including some we’ve seen before. Like the RNGs of most languages, though, Math.random() has its drawbacks. Of these drawbacks, one is very important for game developers: there’s no way to seed! Why is this bad? That cuts to the very heart of random number generation on computers.

True randomness is hard to come by. On a computer, you have a few ways of creating it, such as noise circuits or measurements of user input. This randomness (entropy) is then converted by code into something usable, such as numbers or bits. On Linux and similar systems, there is a special file, /dev/random, that provides access to the random data. Some programming languages then allow developers to use the data through mechanisms like C++’s std::random_device.

All well and good, right? As long as your source has enough randomness, you can generate all the numbers you need. But, contrary to thermodynamics, a computer’s entropy can run out. When that happens, /dev/random stops working, waiting until it can conjure up enough random bits to meet your needs. Other systems fare no better.

Fortunately for us, games don’t often need true random numbers for gameplay. (Authentication and encryption are a different story.) So we can get away with something that only looks random. That’s good for us, because there are plenty of algorithms out there that can make a convincing string of numbers, including Math.random(). Technically, they aren’t “true” random numbers, and they all have a weakness that can allow someone with enough data to guess the next numbers in the sequence. But that very predictability comes in handy for game development. Starting from a little bit of data (from one number to a few, depending on the algorithm used), we get a whole set of numbers, billions of them or more. That starting data is the seed.

Most languages that I’ve used allow you to set the seed on the RNG. C, for example, has the srand() function, while Python provides random.seed(). But JavaScript doesn’t have this. Instead, the RNG is seeded for you when your program/app/page loads, and it will be different every time. Usually, that’s not a problem.

Sometimes, though, you need the predictable sequence that comes from using a seed. Procedural generation is one notable example. Look at Minecraft: a whole world can be created from a simple string of text. Obviously, there’s randomness injected in the process, but it’s made on-demand. Think how hard it would be to store the whole world after creating it. But, if you only had JavaScript’s RNG, you wouldn’t have much of a choice.

There are better RNG implementations out there. Indeed, many have written JS versions of them. Here’s my attempt at the popular algorithm known as xorshift+.

module.exports = (function() {
    // xorshift128+ requires 128 bits of state (we'll seed later)
    var state = new Uint32Array(4);

    // Scaling factor (2^32) to convert Math.random floats into integers
    var MAXINT_PLUS_1 = Math.pow(2,32)

    // Pre-fill the state array (can later be seeded)
    // This is required because xorshift can't have state of all zero
    for (var i = 0; i < state.length; i++) {
        state[i] = Math.random() * MAXINT_PLUS_1;
    }

    // A saved random number, since we're returning 32-bit numbers
    var saved;

    return {
        // Seeds the internal RNG.
        seed: function(s) {
            if (s === 0 || s == null) {
                // Can't use a zero seed (maybe throw an exception?)
                return;
            }

            // TODO Handle various types of arguments (just numbers/arrays for now)
            if (typeof s === 'number') {
                // Use only the lowest 32 bits right now
                state[0] = s >>> 0;
            } else if (s.constructor && s.constructor.name === 'Uint32Array') {
                for (var i = 0; i < state.length; i++) {
                    if (s[i] !== undefined) {
                        state[i] = s[i];
                    } else {
                        state[i] = 0;
                    }
                }
            }
        },

        // Returns a random float between 0 and 1 (exclusive),
        // with 32 bits of precision.
        random: function() {
            // If we already have a saved number, return it,
            // also clearing it for later use.
            if (saved != null) {
                var temp = saved;
                saved = null;
                return temp / MAXINT_PLUS_1;
            }

            // x = s[0]
            var x = new Uint32Array(2);
            x[0] = state[0];
            x[1] = state[1];

            // y = s[1]
            var y = new Uint32Array(2);
            y[0] = state[2];
            y[1] = state[3];

            // s[0] = y
            state[0] = y[0];
            state[1] = y[1];

            // (a): x ^= x << 23
            var xl23 = new Uint32Array(2);
            xl23[0] = x[0] << 23;
            xl23[1] = (x[1] << 23) & (x[0] >> 11);
            x[0] ^= xl23[0];
            x[1] ^= xl23[1];

            // (b): x ^= x >> 17
            var xr17 = new Uint32Array(2);
            xr17[1] = x[1] >>> 17;
            xr17[0] = (x[0] >>> 17) & (x[1] << 19);
            x[0] ^= xr17[0];
            x[1] ^= xr17[1];

            // (c): x ^= y ^ (y >> 26)
            var yr26 = new Uint32Array(2);
            yr26[1] = y[1] >>> 26;
            yr26[0] = (y[0] >>> 26) & (y[1] << 6);
            x[0] ^= y[0] ^ yr26[0];
            x[1] ^= y[1] ^ yr26[1];

            // s[1] = x
            state[2] = x[0];
            state[3] = x[1];

            // return x + y
            var retval = new Uint32Array(2);
            retval[0] = x[0] + y[0];
            retval[1] = x[1] + y[1] + (retval[0] < x[0]);
            saved = retval[1];
            return retval[0] / MAXINT_PLUS_1;
        }
    };
})();

I’ve written it as a Node.js module, but you can easily adapt it to a browser environment by changing module.exports on line 1 to window.xorshift or whatever you like. Whether it’s attached to the global window (browser) or loaded with require() (Node), the function creates an object with two methods: random() and seed(), both of which are explained below.

This isn’t a perfect implementation, and it does have a limitation that the original doesn’t have. This is because of JavaScript’s number handling, which might best be termed as “special”. JS only really has 64-bit floats for numbers, unless you do even more TypedArray contortions than I did here. So I had to compromise by making the random() function output numbers between 0 and 1 with 32 bits of resolution. Each run of the algorithm creates 64 bits of randomness, so I split that into two numbers, saving the second for the next call to the RNG. Changing the function to return integers instead of floats is easy enough: remove the two divisions by MAXINT_PLUS_1.

The whole reason for making this thing was to have a predictable RNG for JavaScript. That’s what the seed() function does. Pass in a single 32-bit integer or a typed array of them, and it will seed the algorithm using that. (One good way to extend this would be to use a hash such as MD5 to allow strings and other objects. That’s why the “TODO” comment is there.) If you don’t, it will use a few numbers generated from Math.random().

Another benefit of this (or any similar RNG) over the stock implementation is that you can create more than one, each tied to its own seed. This means that, for example, you can have your world generator running off a different sequence than your AI. You would then only have to save the seed for the world RNG, while the AI RNG gets reseeded when you reload the game. This would prevent players from, say, repeatedly reloading to get a good outcome of a battle in a strategy game.

As usual, I didn’t make the algorithm; I only wrote the code in this post. You can use it for whatever purpose you like, but I’m sure there are better implementations out there. I didn’t check, mostly because I wanted to try my hand at writing it first. It’s good practice.

Until next time, have fun!

Let’s make a language – Part 1a: Phonology (Intro)

The sound of a language is, in a sense, it’s first impression. And first impressions matter. How a language sounds, the spoken noises that it uses, can certainly influence the opinion of a listener (or reader). In the real world, for example, Westerners often perceive Arabic as a “harsh” language because of its series of “guttural” sounds. We might also talk about Chinese as a “musical” language, since it makes use of tone, a quality we’ll come to later. For conlangs, things are no different. The Elvish languages of Lord of the Rings are praised as melodious, while the Klingons of Star Trek speak a tongue that, like them, comes across as abrasive, violent. (Of course, in the case of conlangs, we have to look at things from the other direction sometimes. Elves have “enchanting” words because they’re supposed to. Klingons are a warrior race, and their language reflects this.)

All this is to say that the sound of your language is important. Even if you’re making a purely written language (like for a book), you might need to pronounce it at some point, and many readers will certainly try. After all, Dothraki began as a few words and phrases scattered almost haphazardly throughout the books of A Song of Ice and Fire. Once those books were turned into the Game of Thrones TV series, Dothraki (and Valyrian, which is barely found in the books at all, apart from a couple of fixed phrases like valar morghulis) had to become something more “real”.

To make a language, we need to understand a little about how languages work, and this is one of those posts. Specifically, we’re looking at what’s called phonology, i.e., the sounds that make up a language. Obviously, if your language isn’t spoken, like a sign language, then this post won’t be of much use. Honestly, though, I have no idea of how to even begin to make a sign language, so that’s the last I’ll say about them. (I can’t think of too many signed conlangs, unless you consider ASL a conlang. The closest thing I can come up with is the elaborate gesturing or “posing” of Daniel Abraham’s Long Price Quartet series, which is more of an addition to speech than a language of its own.) Also, if you’re making a language for aliens that don’t speak the way we do, then you’ve probably got bigger problems than I can solve.

(Digression: Okay, I had this whole thing planned out where I’d go over all the phonology stuff. But I scrapped it. Why? A few reasons. First, it was about 2,000 words just for the section on consonants. That was way too long for a post. Second, plenty of other people have already done the same thing. So, instead, I’ll leave you with a link to Wikipedia’s page on the International Phonetic Alphabet, which has clickable links for just about every possible sound found in human languages, and I’ll turn this post into something more general and useful for a beginning conlanger.)

The Sounds We Make

Every language in the world has a number of phonemes, which are basic units of sound. Think of them as letters, except we’re not necessarily talking about the ones in the alphabet. English, for instance, has 26 letters, but 40 or so phonemes, depending on dialect. Many of these phonemes, however, can surface as slightly different sounds, or allophones. The P sounds in pot and top are good examples of this. They don’t sound exactly the same, but they’re close enough that English speakers call them the same thing. A language like Hindi, on the other hand, does say they are different sounds: /p/ and /pʰ/.

Which (and how many) sounds you use in your language is largely a matter of style, and that directly relates to what kind of conlang you’re making. For languages intended to be for communication (auxlangs), you definitely want to use the most common sounds, most of which have IPA values of basic English letters: /p/, /t/, /k/, and so on. Adding in fancy things like retroflex consonants (despite being common in the very populous Indian subcontinent) or palatalization (found in Slavic languages and Irish, but not many other places) will only make things harder for the speakers that have to learn not only a new language, but new sounds to go with it.

For every other type of conlang, you might think you can just go wild with phonemes. Obviously, you can. I’m not stopping you. But something intended to sound natural should fit the patterns of natural languages. Otherwise, you end up with what I’ve heard called “shotgun phonology”. You may as well throw darts at an IPA chart. So, instead, let’s take a look at what linguistic evolution has come up with, and see if we can make something to match it.

Consonants

We’ll start with consonants, both because there’s more of them and because that’s where some of the most interesting possibilities lie. English has about two dozen, which is pretty much average in the world, according to Chapter 1 of the World Atlas of Language Structures. (By the way, bookmark that site; we’ll be going back to it a lot. I’ll usually refer to it as WALS from here on out.) The minimum is about 6 or so, found in a few Pacific and Amazon languages like Rotokas and Pirahã. The high end goes up to around 80 in the Caucasian language Ubykh, and the click languages of Africa can have even more if you count the combination of click and stop as a single phoneme.

So, anywhere from 6 to 80. That’s quite a range, but we can narrow it down once we start looking for patterns. That’s the key to making a conlang seem natural in its phonemic inventory. Take English as an example, since we’re already using it. English has a set of labial consonants (/p b m f v/), a set of dentals (/t d n s z θ ð l r/), some post-alveolar or palatals (/ʃ ʒ tʃ dʒ j/), and a few velars (/k g ŋ w/). /h/ is the odd one out, but it’s like that in a lot of languages, so that’s okay. Looking at it from the other dimension, English has stops (/p b t d k g/), nasals (/m n ŋ/), fricatives (/f v θ ð s z ʃ ʒ h/), affricates (/tʃ dʒ/), and approximants (/r l j w/). Any way you look at it, essentially every consonant is related to another. There’s not, say, a uvular stop out by itself.

Any language you can think of works the same way. Spanish has a palatal series (/tʃ ɲ ʎ j/), Hindi has a set of retroflex consonants. The languages with smaller consonant inventories have broader distinctions. Rotokas, with its half a dozen consonants, divides them up in two dimensions: voiced or voiceless, and labial, alveolar, or velar. The enormous systems of the Caucasus come about similarly, but making finer distinctions. The 58 consonants of Abkhaz illustrate this. Labialized and non-labialized consonants are different in that language, and there is a set of ejective stops. Both of these combine to increase the inventory while avoiding outliers.

That’s not to say you can’t have outliers. You just need a good reason for them. If you’ve got /p/, /b/, and /t/ already, you’ll probably have /d/, too, but that doesn’t always have to be the case. Especially as you go “down” the phonetic chart, from stops to fricatives to approximants, there are a lot more opportunities to add wrinkles to the system. You can have /s/ and /k/ without having /x/, like English. Or /r/ without /l/, like in Japanese.

The same is true for “rare” sounds. Conlangers tend to over-represent two of these in particular: the English “th” sounds /θ ð/. (I’m guilty of it myself, with my language Suvile.) These sounds are comparatively rare (about 1 in 10 languages have them), but they’re far more common in conlangs. The same is true for some of the more outlandish distinctions, and the reason why is simple. A conlanger sees a sound he likes, and he builds the language specifically to have it, whether it fits or not. Again, if that’s what you like, go for it, but the result might feel “fake”.

Vowels

Vowels have a bit less in the way of possibilities, and vowel systems tend to fall into a few basic categories. Here, English is on the large end of the scale, with up to 20 or more vowel sounds, depending on dialect. A few languages have only two vowel phonemes (Ubykh, mentioned above, is one of these), though these may take on different qualities at different points in a word. Five is the most common, though, according to WALS Chapter 2, and those five are usually the cardinal vowels /a e i o u/. Six is also common, with the addition usually being a schwa (/ə/) or a high central vowel like /ɨ/, though something like /æ/ isn’t out of the question. Systems with four vowels drop one of the cardinal quintet, usually /o/ or /u/. Three-vowel systems are almost always /a i u/, as these are maximally distinct.

Like with consonants, the key here is regularity, at least at the start. The common five vowels can be split into high (/i u/), middle (/e o/), and low. Or you could divide them into front (/i e/), central (/a/), and back (/o u/). Larger vowel systems become that way because they add dimensions. If you have the front vowels /i/ and /e/ and the rounded vowels /o/ and /u/, it’s not that much of a stretch to add in the front and rounded /y/ and /ø/. Similarly, a quality like length or nasalization tends to “spread” through the vowel system, multiplying the number of phonemes.

Vowel harmony is another of those ideas that conlangers get carried away with. The canonical example is Turkish, with its eight vowels /i y ɯ u e ø o a/. This makes a kind of 3D grid, where each vowel is either front or back, either high or low, and either rounded or unrounded. Turkish grammatical suffixes come in different forms, depending on which type of vowel they need, and a word must have its vowels all front or all back. This has an appealing symmetry of the kind that conlangers tend to love. Like the consonantal rarities above, though, there needs to be a reason, even if that reason boils down to “because it sounds cool”.

In my opinion, if you have no other pressing needs (like fitting in with names you’ve already made, for instance) then you should probably start with the basic five vowels. If you’re making an auxiliary language, then I’d strongly suggest stopping there. (Volapük used front vowels, because its creator was German. Esperanto went with the basic set instead. Which one’s more popular?)

Everybody else probably needs more, though. Still, start with the basics. If you add vowels, make sure they fit. More than consonants, vowels have a tendency to shift around in speech, almost like they’re floating. They like to be as distinct as possible. Sure, it might sound fun to have a language whose vowels are /i y e ø ɨ ʉ ɛ ɔ ɜ ɑ/, but it wouldn’t stay that way for long. A couple of generations of real language evolution would turn it into something like /i ɪ e ə æ u ʊ o/.

Tone

Besides consonants and vowels, we have one more thing to add to our study of phonology. Tone is probably the most popular in conlangs, simply because it isn’t found in many languages Westerners would be familiar with, making it seem exotic. (And the one major tonal language group is Chinese, further reinforcing that stereotype.) But tone is actually quite common in the world’s languages, especially in places of high linguistic concentration like Africa and the Amazon.

Tone itself can be divided into two varieties. Mandarin Chinese is an example of the first, which uses relative changes in pitch: level (called “high” in studies of the language), rising, dipping (falling from a low pitch to an even lower one, then sometimes rising again), falling, and a fifth, neutral tone found in weak syllables. Other languages have more or less complicated systems, but the idea remains the same: it’s the change in pitch that is important.

The alternative is a system where the tones themselves are steady, but at different levels. This is found, e.g., in Bantu languages of Africa. These are usually languages with two tones, a high and a low, or three, adding a middle tone. Four or more tones of this kind are rare, and it’s easy to see why. I mean, you could make a language with seven tones, each corresponding to a note on the major scale, and such a thing has indeed been done, but it would be awfully hard to speak. For speakers of such a language, singing lessons might be an integral part of grammar classes!

Obviously, an international auxlang likely won’t have tone, although one intended solely for communication in places where most languages are already tonal wouldn’t be out of the ordinary. For the more artistic conlangs, do whatever you want! In terms of numbers of languages, about half are tonal, though this is skewed by the large concentrations of tonality I mentioned above. (On a personal note, I’ve made one serious attempt at a tonal language, Lyssai. It’s for a race of elf-like forest dwellers in a story I’ll eventually write.)

Flavor

Note: If you’re making an auxiliary language, you can probably skip this section.

A lot of the flavor of a language comes from its sound, and that sound comes largely from the phonemes used in the language. (Some of it comes from the syllable structure and stress patterns, which we’ll get into next time.) Guttural sounds from the back of the throat grate on American ears, while the liquid sounds of approximants and trills feel soft. Palatalized sounds have a “slurring” quality, while dentals make us think of a lisp.

For fictitious cultures, this stereotyping becomes useful. Tolkien puts into the mouths of his elves words full of fricatives and approximants and voiceless stops, all phonemes perceived as soft. In sharp contrast, orc speech is full of aspirated or voiced stops, both “uglier” types of sounds, a subtle way of confirming their status as the enemy.

Of course, if you’re making a language meant to be spoken by actors, you need to take that into account, too. That’s why Dothraki, for example, has such a relatively simple phonology. (The exception is the lone uvular stop [q], which goes against what I said earlier about phoneme sets, but he’s getting paid, and I’m not. Oh well.)

So, the lessons we can learn here are many:

  1. If you’re making an auxiliary language, choose sounds and sound distinctions that are fairly common. Esperanto arguably screwed up by including a palatal series. Volapük did the same with front rounded vowels. Of course, French was once the lingua franca (it’s right there in the name), and it has a pretty complex phonology, so there are always exceptions.

  2. Artistic languages can have whatever sounds you can pronounce. But remember your audience. Americans probably aren’t going to be able to pronounce pharyngeals. Japanese speakers might not be able to manage [θ] and [ð].

  3. Phonemes, especially stops, tend to be connected. A distinction made on only one phoneme feels unnatural. It’s not impossible, mind you, just less likely.

  4. Vowels are like a gas. They expand to fill their space, and they spread out. The fewer you have, the more guises they can take. A language with only /a i u/, for example, can still have [e o] as allophones.

  5. Tone is nice, and it can be interesting, but you need to study up on how it’s used. (Actually, this can go for anything else in this gigantic post.)

  6. There are more things on heaven and earth than are dreamt of in your language. The conlang community has a saying known as ANADEW: a natlang (natural language) already did, except worse. Almost every concept that a conlanger thought he came up with, some real language spoken somewhere has it.

That’s it for now. (Finally!) Next time, we’ll get into the sound systems of our two languages, Isian and Ardari.

First Languages for Game Development

If you’re going to make a game, you’ll need to do some programming. Even the best drag-and-drop or building-block environment won’t always be enough. At some point, you’ll have to make something new, something customized for your on game. But there are a lot of options out there. Some of them are easier, some more complex. Which one to choose?

In this post, I’ll offer my opinion on that tough decision. I’ll try to keep my own personal feelings about a language out of it, but I can’t promise anything. Also, I’m admittedly biased against software that costs a lot of money, but I know that not everyone feels the same way, so I’ll bite my tongue. I’ll try to give examples (and links!) of engines or other environments that use each language, too.

No Language At All

Examples: Scratch, Construct2, GameMaker

For a very few cases, especially the situation of kids wanting to make games, the best choice of programming language might be “None”. There are a few engines out there that don’t really require programming. Most of these use a “Lego” approach, where you build logic out of primitive “blocks” that you can drag and connect.

This option is certainly appealing, especially for those that think they can’t “do” programming. And successful games have been made with no-code engines. Retro City Rampage, for example, is a game created in GameMaker, and a number of HTML5 mobile games are being made in Construct2. Some other engines have now started creating their own “no programming required” add-ons, like the Blueprints system of Unreal Engine 4.

The problem comes when you inevitable exceed the limitations of the engine, when you need to do something its designers didn’t include a block for. For children and younger teens, this may never happen, but anyone wanting to go out of the box might need more than they can get from, say, Scratch’s colorful jigsaw pieces. When that happens, some of these engines have a fallback: Construct2 lets you write plugins in JavaScript, while GameMaker has its own language, GML, and the newest version of RPG Maker uses Ruby.

Programming, especially game programming, is hard, there’s no doubt about it. I can understand wanting to avoid it as much as possible. Some people can, and they can make amazing things. If you can work within the limitations of your chosen system, that’s great! If you need more, though, then read on.

JavaScript

Examples: Unity3D, Phaser

JavaScript is everywhere. It’s in your browser, on your phone, and in quite a few desktop games. The main reason for its popularity is almost tautological: JavaScript is everywhere because it’s everywhere. For game programming, it started coming into its own a few years ago, as mobile gaming exploded and browsers became optimized enough to run it at a decent speed. With HTML5, it’s only going to get bigger, and not just for games.

As a language, JavaScript is on the easy side, except for a few gotchas that trip up even experienced programmers. (There’s a reason why it has a book subtitled “The Good Parts”.) For the beginner, it certainly offers the easiest entry: just fire up your browser, open the console, and start typing. Unity uses JS as its secondary language, and about a million HTML5 game engines use it exclusively. If you want to learn, there are worse places to start.

Of course, the sheer number of engines might be the language’s downfall. Phaser might be one of the biggest pure JS engines right now, but next year it could be all but forgotten. (Outside of games, this is the case with web app frameworks, which come and go with surprising alacrity.) On top of that, HTML5 engines often require installation of NodeJS, a web server, and possibly more. All that can be pretty daunting when all you want to do is make a simple game.

Personally, I think JavaScript is a good starting language if you’re careful. Would-be game developers might be better off starting with Unity or Construct2 (see above) rather than something like Phaser, though.

C++ (with a few words on C)

Examples: Unreal Engine 4, SFML, Urho3D

C++ is the beast of the programming world. It’s big, complex, hard to learn, but it is fast. Most of today’s big AAA games use C++, especially for the most critical sections of code. Even many of the high-level engines are themselves written in C++. For pure performance, there’s not really any other option.

Unfortunately, that performance comes at a price. Speaking as someone who learned C++ as his second programming language, I have to say that it’s a horrible choice for your first. There’s just too much going on. The language itself is huge, and it can get pretty cryptic at times.

C is basically C++’s older brother. It’s nowhere near as massive as C++, and it can sometimes be faster. Most of your operating system is likely written in C, but that doesn’t make it any better of a choice for a budding game programmer. In a way, C is too old. Sure, SDL is a C library, but it’s going to be the lowest level of your game engine. When you’re first starting out, you won’t even notice it.

As much as I love C++ (it’s probably my personal favorite language right now), I simply can’t recommend starting with it. Just know that it’s there, but treat it as a goal, an ideal, not a starting point.

Lua

Examples: LÖVE, many others as a scripting or modding language

Lua is pretty popular as a scripting language. Lots of games use it for modding purposes, with World of Warcraft by far the biggest. For that reason alone, it might be a good start. After all, making mods for games can be a rewarding start to game development. Plus, it’s a fairly simple language that doesn’t have many traps for the unwary. Although I’ll admit I don’t know Lua as well as most of the other languages in this list, I can say that it can’t be too bad if so many people are using it. I do get a kind of sense that people don’t take it seriously enough for creating games, though, so take from that what you will.

C#

Examples: Unity3D, MonoGame

C# has to be considered a good candidate for a first language simply because it’s the primary language of Unity. Sure you can write Unity games in JavaScript, but there are a few features that require C#, and most of the documentation assumes that’s what you’ll be using.

As for the language itself, C# is good. Personally, I don’t think it’s all that pretty, but others might have different aesthetic sensibilities. It used to be that C# was essentially Microsoft-only, but Mono has made some pretty good strides in recent years, and some developments in 2015 (including the open-sourcing of .NET Core) show positive signs. Not only that, but my brother finds it interesting (again, thanks to Unity), so I almost have to recommend at least giving it a shot.

The downside of C# for game programming? Yeah, learning it means you get to use Unity. But, that’s about all you get to use. Besides MonoGame and the defunct XNA, C# doesn’t see a lot of use in the game world. For the wider world of programming, though, it’s one of the standard languages, the Microsoft-approved alternative to…

Java

Examples: LibGDX, JMonkeyEngine, anything on Android

Java is the old standard for cross-platform coding. The Java Virtual Machine runs just about anywhere you can think of, even places it shouldn’t (like a web browser). It’s the language of Minecraft and your average Android app. And it was meant to be so simple, anybody could learn it. Sounds perfect, don’t it?

Indeed, Java is simple to learn. And it has some of the best tools in the world. But it also has some of the slowest, buggiest, most bloated and annoying tools you have ever had the misfortune of using. (These sets do overlap, by the way.) The language itself is, in my opinion, the very definition of boring. I don’t know why I feel that way, but I do. Maybe because it’s so simple, a child could use it.

Obviously, if you’re working on Android, you’re going to use Java at some point. If you have an engine that runs on other platforms, you might not have to worry about it, since “native” code on Android only needs a thin Java wrapper that Unity and others provide for you. If you’re not targeting Android, Java might not be on your radar. I can’t blame you. Sure, it’s a good first language, but it’s not a good language. The me from five years ago would never believe I’m saying this, but I’d pick C# over Java for a beginning game developer.

Python

Examples: Pygame, RenPy

I’ll gladly admit that I think Python is one of the best beginner languages out there. It’s clean and simple, and it does a lot of things right. I’ll also gladly admit that I don’t think it can cut it for game programming. I can say this with experience as I have tried to write a 2D game engine in Python. (It’s called Pyrge, and you can find the remnants of it on my Github profile that I won’t link here out of embarrassment.). It’s hard, mostly because the tools available aren’t good enough. Python is a programmer’s language, and Pygame is a wonderful library, but there’s not enough there for serious game development.

There’s always a “but”. For the very specific field of “visual novels”, Python does work. RenPy is a nice little tool for that genre, and it’s been used for quite a few successful games. They’re mostly of the…adult variety, but who’s counting? If that’s what you want to make, then Python might be the language for you, just because of RenPy. Otherwise, as much as I love it, I can’t really recommend it. It’s a great language to learn the art of programming, but games have different requirements, and those are better met by other options.

Engine-Specific Scripting

Examples: GameMaker, Godot Engine, Torque, Inform 7

Some engine developers make their own languages. The reasons why are as varied as the engines themselves, but they aren’t all that important. What is important is that these engine-specific languages are often the only way to interact with those environments. That can be good and bad. The bad, obviously, is that what you learn in GML or GDScript or TorqueScript doesn’t carry over to other languages. Sometimes, that’s a fair trade, as the custom language can better interact with the guts of the engine, giving a performance boost or just a better match to the engine’s quirks. (The counter to this is that some engines use custom scripting languages to lock you into their product.)

I can’t evaluate each and every engine-specific programming language. Some of them are good, some are bad, and almost all of them are based on some other language. Godot’s GDScript, for example, is based on Python, while TorqueScript is very much a derivative of JavaScript. Also, I can’t recommend any of these languages. The engines, on the other hand, all have their advantages and disadvantages. I already discussed GameMaker above, and I think Godot has a lot of promise (I’m using it right now), but I wouldn’t say you should use it because of its scripting language. Instead, learn the scripting language if you like the engine.

The Field

There are plenty of other options that I didn’t list here. Whether it’s because I’m not that familiar with the language, or it doesn’t see much use in game development, or because it doesn’t really work as a first language, it wasn’t up there. So here are some of the “best of the rest” options, along with some of the places they’re used:

  • Swift (SpriteKit) and Objective-C (iOS): I don’t have a Mac, which is a requirement for developing iOS apps, and Swift is really only useful for that purpose. Objective-C actually does work for cross-platform programming, but I’m not aware of any engines that use it, except those that are Apple-specific.

  • Haxe (HaxeFlixel): Flash is dying as a platform, and Haxe (through OpenFL) is its spiritual successor. HaxeFlixel is a 2D engine that I’ve really tried to like. It’s not easy to get into, though. The language itself isn’t that bad, though it may be more useful for porting old Flash stuff than making new games.

  • Ruby (RPG Maker VX Ace): Ruby is one of those things I have an irrational hatred for, like broccoli and reality shows. (My hatred of cats, on the other hand, is entirely rational.) Still, I can’t deny that it’s a useful language for a lot of people. And it’s the scripting language for RPG Maker, when you have to delve into that engine’s inner workings. Really, if you’re not using RPG Maker, I don’t see any reason to bother with Ruby, but you might see things differently.

  • JavaScript variants (Phaser): Some people (and corporations), fed up with JavaScript’s limitations, decided to improve it. But they all went in their own directions, with the result of a bewildering array of languages: CoffeeScript, TypeScript, LiveScript, Dart, and Coco, to name a few. For a game developer, the only one directly of any use is TypeScript, because Phaser has it as a secondary language. They all compile into JS, though, so you can choose the flavor you like.

If there’s anything I missed, let me know. If you disagree with my opinions (and you probably do), tell me why. Any other suggestion, criticism, or whatever can go in the comments, too. The most important thing is to find something you like. I mean, why let somebody else make your decisions for you?

Let’s make a language – Introduction

On the surface, the title of this post sounds ludicrous. Make a language? How could anyone do that? But people have done it. I’m one of them. And this series will (I hope) help you to do the same. In the end, you should have all the knowledge needed to make your own constructed language (or conlang).

Why Make a Language?

I know, it doesn’t exactly sound like something a normal person would do, but there are reasons. They might not be good reasons, but they’re still reasons. So, why would you want to make your own language? Let me count the ways:

  1. Worldbuilding. You’re an author (or a screenwriter or game developer) and you need something more than just gibberish. Sci-fi has aliens, fantasy has elves, and even Hollywood action movies might want to have the bad guys speak in something other than obvious Arabic or Russian. This is, in my opinion, the most important reason, and the one that will be the main focus of this series of posts. Examples of “worldbuilding” conlangs include Tolkien’s Sindarin (as seen in Lord of the Rings), Avatar‘s Na’vi, and the Dothraki language of Game of Thrones.
  2. Communication. The earliest attempts at created languages were mostly made to ease communication between speakers of multiple, indistinct tongues. In effect, they were trying to make their own lingua franca. That sort of thing still goes on (now usually called an “auxiliary language”, sometimes shortened to auxlang). Esperanto is the most famous example of this class of conlang, but it also includes Lojban and older efforts such as Ido and Novial.
  3. Art and philosophy. Some languages are created purely for their artistic effect, or specifically engineered to some ideal. Either way, they aren’t necessarily intended to be spoken. Rather, they’re more to be admired. The language Toki Pona fits into this class, as it was specifically designed as a kind of experiment in minimalism, while Ithkuil forms an almost perfect counterpart of extreme complexity.
  4. Secrecy. Writing down your thoughts in a form only you can understand certainly has its uses. After all, if you’re the only one who can read the language, then it’s effectively not much different from a one-time pad, right? (Well, not exactly. First, it probably won’t be much better than a cryptogram, since you’ll want something that’s easy for you to learn. Second, your notes will be as good as a key. Still, it might be fine for a diary or journal or something like that.) Obviously, there aren’t any good examples of a language like this.
  5. Fun. We don’t always need a reason to do things. Most conlangs are made because their creators wanted to make them. That includes most of my early efforts, for example. (I’d link to them, but they were never online to begin with.) Plus, it’s a good way to learn. Case in point: I hated English in school. Absolutely loathed it. Didn’t really care too much for Spanish in high school, either. Now, I’m writing this post, and I wouldn’t have done that if I hadn’t tried to make a language a long time ago. In the past 15 years, I’ve probably learned more about things like phonology, language evolution, and grammar in my spare time than many college graduates would pick up in a university setting (excluding those that major in linguistics, obviously).

What Are We Going To Do?

Well, the way I’ve planned it, the title of this post is a bit of a fib. We’re not going to make a language. We’re going to make two of them, running in parallel.

Language #1 is going to be the simpler, more familiar one. It’ll be a bit like English, with a lot of other influences, especially the top languages of Europe. There won’t be much here in the way of weird grammar or sounds that make you feel like throwing up when you try to pronounce them. We’ll call this language Isian.

The second language will be a bit more…advanced. Here, we can throw in odd sounds, strange words, and concepts that might boggle the mind of the average speaker of American English. It won’t be too far out there, and it won’t hold a candle to some of the real-world languages found in remote parts of Africa, the Amazon, or New Guinea, but it will be unlike any of the choices you probably had in high school. This language will be called Ardari.

For both languages, before we do anything, we’ll start with a little bit of theory for the bit of creating that we’re doing. For example, the first part of the series will be about phonology, so I’ll make a post that delves into the science of phonology and talks about how that relates to conlangs in general. That will be followed by a post where we create the sound system of Isian, then another that does the same for Ardari. Sometimes, if it’s a particularly small bit of info, I’ll combine both languages into a single post.

The Home Game

At any point along the way, comments are welcome, as are corrections and (constructive) criticism. This will be a bit of a democratic effort. (In other words, I’ll take all the help I can get!) And, of course, you’re perfectly welcome to play along at home, making your own conlang as we go. If you do, I’d love to see it, so don’t be afraid to post!

2D Grid Movement with Kinematic Bodies (Godot)

Movement on a grid is common in many games, especially 2D games. In one of my current projects (a “falling blocks” game), this particular problem came up: how do you get grid-based (or discrete) movement on the X-axis while retaining free, continuous movement on the Y-axis? Specifically, I’m using the Godot engine, but the same principle should carry over to any game engine or development environment.

Movement in 2D

Many 2D game engines offer physics systems, and they all tend to be pretty similar (probably because most of them use Box2D under the hood). While your game may be all about sprites, the physics code works with bodies and shapes. Roughly speaking, bodies represent the “mass” of your game objects, while a body’s shapes outline its area. When two bodies’ shapes overlap, there’s a collision, which is handled however your game is supposed to: kill an enemy, take damage from a bullet, etc.

Depending on the specific engine, you have a few different kinds of shapes available. Godot, for example, lets you assign rectangles, circles, lines, “capsules” (like a rectangle with rounded caps on each end), and general polygons. If these aren’t enough, you can combine multiple shapes on a single body. Of course, most 2D engines work this way, so you probably already knew all that.

For bodies, you again have options. Walls and other immobile obstacles are usually static bodies (i.e., they don’t move), and interactive elements are often rigid bodies fully under the influence of physics. The player character, in many engines, is a third type of body, the kinematic body, which causes collisions and stops when it hits a static body, but isn’t affected by forces or friction or, indeed, any physics at all. Once again, though, you already know all of this, because that’s how most 2D physics engines work.

The Setup

For this specific problem, I’m using a kinematic body to represent each falling block. Attached to that body is a sprite (the default Godot icon, for this post) and a collision shape, as you can see in this screenshot:

KinematicBody2D scene tree

(In Godot, there are separate classes for 2D and 3D physics objects, so we have to use KinematicBody2D and CollisionShape2D.)

The kinematic body is the basic object representing each block, the sprite is its appearance, and the collision shape defines its area. Simple enough. Now, what we want to happen is this: move the sprite in two different ways. On the Y-axis, the block should fall down continuously, moving through every point on its way to the bottom. On the X-axis, however, we want the block to “jump” from one position to another, because the blocks have to stack perfectly.

I’ve also set up a scene to use as a base. It’s nothing much, just walls on either side and a floor on the bottom of the screen:

Scene tree for walls and floors

When all this is done, we’ll have a sprite falling from the top of the screen until it hits the “wall” at the bottom. At any point after it appears, you can click and drag it to move it from side to side, and it will stay on a grid, something like this:

Grid movement

Making the Body

We can make the body/sprite/shape combination as follows:

  • The KinematicBody2D is the root node. The only property I changed was reducing the collision margin (Collision > Margin in the Inspector window) to 0.001, the lowest it can go. You don’t actually need to do this for the example, but it may help if you have a problem with collisions detected when bodies aren’t really touching. (There’s also a script attached to this node, but we’ll get to that.)
  • The Sprite is our image. Load the icon.png file that’s included with every Godot project, and you can leave pretty much everything else as is.
  • The CollisionShape2D node, as you might expect, is our collision shape. Due to the way Godot works, we need to define the shape of the shape, which you can do under CollisionShape2D > Shape in the Inspector. Create a new RectangleShape2D in the menu, and set its X and Y extents to something a little less than 32:

RectangleShape2D Properties

(The logo image is 64×64 pixels in size, and extents are measured from the center. If we set the extents to exactly 32, then some blocks might be considered colliding when they really aren’t. That’s because of the collision margin I mentioned above. You can even like 31.999 if you like, and that may work better than 31. Honestly, I’m not sure at the moment.)

The Code

Now that we have all that out of the way, we come to the real meat of the post, the code. Add a new script to your KinematicBody2D node. I named mine gridmove.gd, but you can call it whatever you want. Anyway, here’s the code:

extends KinematicBody2D

# Our accumulated motion on the X axis
var xaccum

# Track if we're dragging a sprite
var mouse_down

# These are the width and height of the sprite
var twidth
var theight

# A default fall speed (like gravity, but velocity instead of acceleration)
const STARTING_SPEED = 100.0

# A velocity vector that we'll use for calculations below
var velocity = Vector2()

func _ready():
    # This object will use input and fixed-timestep physics
    set_process_input(true)
    set_fixed_process(true)

    # Initialize our variables
    xaccum = 0
    twidth = get_node("Sprite").get_texture().get_width()
    theight = get_node("Sprite").get_texture().get_height()
    mouse_down = false
    velocity.y = STARTING_SPEED

func _fixed_process(delta):
    # The object will fall until it hits the bottom of the world or another object
    var motion = velocity * delta

    # Test if we've accumulated enough movement to "jump" one grid square,
    # If we have, then we'll add that much movement to our motion vector.
    if abs(xaccum) > twidth:
        motion.x = twidth * sign(xaccum)
        xaccum -= twidth * sign(xaccum)
    else:
        motion.x = 0

    # Move the object as much as possible
    motion = move(motion)

    # If we're colliding (with the wall or another object), 
    # then we need to modify our motion vector.
    # See the Godot wiki for how and why this works:
    # https://github.com/okamstudio/godot/wiki/tutorial_kinematic_char#problem
    if is_colliding():
        var n = get_collision_normal()
        motion = n.slide(motion)
        move(motion)

    # If the mouse button has been released,
    # we can stop worrying about motion on the X axis
    if not mouse_down:
        xaccum = 0

func _input(event):
    # Create a rectangle covering the entire sprite area
    var gp = get_global_pos()
    gp.x -= twidth/2
    gp.y -= theight/2
    var gr = Rect2(gp, Vector2(twidth, theight))

    # If the left mouse button is pressed while over the object,
    # all we do is set our state variable. If it's released anywhere,
    # we clear that same variable.
    if event.type == InputEvent.MOUSE_BUTTON and event.button_index == 1:
        if gr.has_point(event.pos):
            mouse_down = event.pressed
            get_tree().set_input_as_handled()
        elif mouse_down and not event.pressed:
            mouse_down = false

    # If the user drags while holding the left mouse button,
    # that's our signal to start accumulating motion.
    if event.type == InputEvent.MOUSE_MOTION and mouse_down:
            xaccum += event.relative_x
            get_tree().set_input_as_handled()

The comments tell you most of what’s going on in the code itself. Basically, what we’re doing is “saving up” the motion on the X-axis until it’s enough to move by one grid “square”, which is the width of the logo sprite. The xaccum variable holds how much motion we’ve saved, and we check it each frame (technically, each physics update period, which isn’t necessarily tied to the frame rate). If we’ve saved up enough, then we move the sprite, deducting that motion from our accumulated value.

The added wrinkle is due to gravity, as you can see at the top of the _fixed_process function. Blocks in this particular scene fall at 100 pixels per second, and then they might move on the X-axis. With a vector, we can represent both of these motions, as in line 44, but then we have a problem. Kinematic bodies, remember, can cause collisions when they move, and the move() method stops when the body collides with another, as explained in the wiki article linked on line 49, which also shows how to use the slide() method to change the motion vector.

Spawning the Blocks

The following script should be added to the root Node of the other scene (the one where we defined the walls and floor). All it does is spawn a new block (body, sprite, and shape) whenever you press Space.

extends Node

var block

func _ready():
    set_process_input(true)
    randomize()
    block = load("res://block.xscn")
    spawn(randi() % 10)

func _input(event):
    if event.type == InputEvent.KEY and event.is_pressed() and event.scancode == KEY_SPACE:
        spawn(randi() % 10)

func spawn(column):
    var node = block.instance()
    var tex = node.get_node("Sprite").get_texture()

    # Add 1 to the column value for the left wall,
    # add 0.5 because positions are relative to the center of an object
    var spawn_x = (column + 1.5) * tex.get_width()
    node.set_pos(Vector2(spawn_x, tex.get_height() / 2))
    add_child(node)

Most of this is basic Godot engine stuff like creating a node instance. We do add a hint of uncertainty by spawning each new block in a random grid column.

Conclusion

There’s a lot more that can be done with this code, and it’s probably not bug-free. There may even be a better way of going about this particular problem. If so, I’d love to hear about it! Also, even though I used Godot for this example, the same pattern will work anywhere you have 2D physics, from big names like Unity, to Phaser and other “simpler” engines. You might even be able to adapt it to work in 3D, but I haven’t really tried. Let me know what you come up with, and have fun!

Random Number Distributions (JS)

Last week, I talked about a method for choosing random values within a set, one of the many uses for random numbers in a game. This time, I’ll go deeper into the notion of probability distributions, and how you can make use of them in game development and other kinds of programming. A word of warning: this post will have code (usually Javascript) and might use a lot of math!

Simply Random

The uniform distribution is the one you already know. It’s behind all the simple RNG functions we saw before, like C’s rand() or Javascript’s Math.random(), and it’s what you get when you roll a single die: every number has an equal chance of coming up. In math terms, if there are n possibilities, then the chance of getting any specific one is 1/n. Couldn’t be simpler, really. For most game uses, this is your main tool, and the “weighted set” from the other post is a handy add-on.

The Bell Curve

I mentioned the bell curve and the normal distribution last time, but here I’ll go into a little bit more detail. The bell curve, of course, is often an early introduction to the world of statistics and terms like “standard deviation”, and (as we saw in the other post) it’s what you start getting when you roll more and more dice at a time.

Obviously, that’s one way to get a normal distribution: roll a bunch of dice and add them together:

var dieRoll = function() { return Math.floor(Math.random() * 6) + 1; }
var bellCurveRoll = 0;
for (var i = 0; i < 10; i++) {
    bellCurveRoll += dieRoll();
}

This gives us something close (though not perfect): random numbers will range from 10 to 60, with 35 being the most common. If we need something better (i.e., more accurate), we’ll need to delve into probability theory. Don’t worry, it’s only a short dive.

Side Quest

The mean might be familiar to you, since it’s not much more than another name for “average”. For a normal distribution, the mean is the center point of the curve, the part where it’s at its highest.

Less familiar is the standard deviation, although you may remember that from high school or early college. It’s important in a lot of stats-based fields, because it measures how “clustered” a group of data points is.

Now, for a set of data, you can calculate the mean and standard deviation. That’s pretty much how things like grading curves work: you take the data and make a distribution that fits. For RNGs, however, we have to work backwards. Instead of calculating the mean and standard deviation for the data, we choose them at the start, and our RNG uses them to provide the proper distribution of values. For the normal distribution, this means that about 68% of the values will be within 1 standard deviation of the mean, 95% within 2, and 99.7% within 3. (By the way, the standard deviation is usually identified in math by the Greek letter sigma: σ. “3-sigma” confidence, then, is 99.7% likelihood that something is true. “Six sigma” encompasses everything within 6 standard deviations of the mean, or about 99.9999998%; it’s not quite “one in a billion”, but it’s pretty close.)

Back to Normal

So, knowing all this boring math, you can start getting your random numbers. Here’s one way of doing it in Javascript:

function normalDistribution (mu, sigma) {
    var u1 = Math.random();
    var u2 = Math.random();

    var z0 = Math.sqrt(-2.0 * Math.log(u1)) * Math.cos(Math.PI*2 * u2);
    var z1 = Math.sqrt(-2.0 * Math.log(u1)) * Math.sin(Math.PI*2 * u2);

    return z0 * sigma + mu;
}

This uses a method called the Box-Muller transform to generate a random number on a bell curve. (The astute reader will notice that the function actually generates two random numbers, but throws one of them away. If you like, you can make a better function that stores the second value and returns it as the next random number.) The two parameters are our mean (mu, because it is often written as the Greek letter μ) and the standard deviation (sigma). The Wikipedia link above explains the theory behind this method, as well as giving links to other ways, some of which might be faster.

Exponential Randomness

Normal distributions have their uses, and they’re about as far as most people get in studying randomness. But we won’t stop there. We’ll move on.

Next up is the exponential distribution. This one isn’t that hard to make in code:

function expoDistribution (lambda) {
    return -Math.log(1.0 - Math.random()) / lambda;
}

But you might wonder if it’s really useful. Well, as it turns out, it does come in handy sometimes. Basically, the exponential distribution can be used anywhere you need a “time between events” type of randomness, like random events firing in a strategy game.

The lambda parameter is what’s called a “rate parameter”, and it’s intimately related to the mean; in fact, it’s the reciprocal: the exponential distribution’s mean is 1 / lambda. But what does it mean? Let’s take an example: say you’re making a game where special power-ups appear, on average, twice every minute. Using the above function with lambda as 2, the result would be the time (in minutes) until the next power-up. (The mean would then be 1/2, or 30 seconds, which makes intuitive sense.)

Pareto, Principle, and Power

If you’ve ever heard of the “Pareto Principle” or the “80-20 rule”, then you’ve got a head start on learning about the Pareto distribution. The general idea is thus: there are a few with a lot, and a lot with a little. “The top 20% control 80% of the wealth” is the most common way to state this particular idea, but a random distribution with these properties can be surprisingly useful in games. Here’s the code:

function paretoDistribution (minimum, alpha) {
    var u = 1.0 - Math.random();
    return minimum / Math.pow(u, 1.0 / alpha);
}

We have two parameters we can control this time. minimum is the lowest possible value that can be returned, while alpha controls the “shape” of the distribution. (Generally, higher values of alpha have a faster drop-off, meaning that lower values have higher probability. The “80-20” distribution has an alpha of log(5) / log(4), or about 1.161.)

The Pareto distribution isn’t just used for wealth, though. Anywhere you have a “long tail”, it might be what you’re looking for. Random city sizes (or star sizes, for an interstellar game) follow a Pareto distribution, and it might be a good way to model “loot drops” in an RPG; less powerful objects are far more common than the Ultimate Sword of Smiting, after all.

In Closing

These aren’t all that’s available, but the four distributions I’ve shown are probably the most useful for programmers, especially game developers. As before, I didn’t originally come up with any of this; it’s all been known since before I was born. Some code is converted from Python’s extensive random module, specifically the functions random.expovariate() and random.paretovariate(). The code for the normal distribution is my Javascript conversion of the Box-Muller transform example at the Wikipedia link above. (By the way, if you want me to post examples for a different language, just ask!)

A lot of people already know all this material, and there are plenty of other sites detailing them better than I can, but I hope that this is enough to get you interested.

Weighted Random Choices (JS)

In programming, we often need to generate random numbers, especially for games. Almost any game will use at least some kind of random number generator (RNG), and most need lots of them to drive the AI, make new levels, and so on.

Any programming language worth using (for games, anyway) will have some way of making an RNG. C has rand(), Javascript has Math.random(), and so on. Game engines usually add in their own ways, like Unity’s Random.value, constructed out of the base provided by whatever language they’re written in. All of these work in basically the same way: you ask the RNG for a value, and it gives you one. Usually, it’s a floating-point number between 0 and 1, which is good if that’s what you need.

Beginners starting out in game development quickly learn how to turn the float value from 0 to 1 (not so useful) into a number in the right range (more useful). The code for rolling a regular, six-sided die usually looks something like this (in Javascript):

var roll = Math.floor(Math.random() * 6) + 1;

It’s a pretty standard technique, and many languages and game engines now have functions that do this for you. And, again, if that’s what you need, then it’s perfect. Sometimes, though, it’s not what you need. This post is for one of those times. Specifically, the case where you need to choose one value from a list.

Simple Choices

When you simply need to pick a single value out of a list (array, vector, whatever), that’s easy. All you’re really doing is the same thing as rolling a die:

var index = Math.floor(Math.random() * array.length);
var choice = array[index];

In effect, we’re choosing a random array index. Easy. Like rolling a die, it gives you an equal chance for each possible value. (If your array contains the numbers from 1 to 6, then you’ve just created a less efficient method of rolling a die.)

Harder Choices

Not everything has an equal probability, though. (Statistics as a science wouldn’t exist if it did. Whether that’s a good or bad thing depends on the way you look at it, I guess.)

Some things can be approximated with the die-rolling method above. Lotteries, for example, work the same way, as do raffles. For some games, that may be all you really need. But other games require more from their RNGs than this.

Take the case of rolling multiple dice. Anybody who has played a role-playing game, or craps in a casino, or even Monopoly knows that, while every number on a die has an equal chance of popping up, things get more complicated when you add more dice. With two dice, for example, a total of 7 is more common than a 2 or 12, because there are more ways to roll numbers that add up to 7: 1+6, 6+1, 2+5, 5+2, 3+4, and 4+3. Add more dice, and the numbers get bigger, the range gets wider, but the idea stays the same: numbers “in the middle” are more likely than those on the outside. (Due to what’s called the central limit theorem, the more dice you roll, the more the graph of possible outcomes starts to resemble a bell curve.)

Rolling a lot of dice is impractical, even on a computer. The probabilities aren’t always so nice and neat that a bell curve works. Maybe you need to choose from a set where the ends are more common than the middle, or a list with a weird distribution of frequencies like the letters in English text. (One out of every eight letters, on average, is E, but Z is over a hundred times less common.) A word game, for instance, would certainly need to do something like this.

Now, there are plenty of different ways of generating random numbers based on frequencies. Here, I’m only going to describe what I think is the simplest. First, we need a problem. Let’s say you’re making a word game that, for whatever reason, uses the letter tiles from Scrabble. (You probably wouldn’t be making an actual Scrabble game, because some people or lawyers might not like that, but we’ll say you’re using the letter frequencies.) Looking at that link, you can probably see that just using random choices won’t help you. We need something different.

First things first, let’s define the set we’re choosing from (note the space at the end, which represents the blank tiles):

var letters = ['a','b','c','d','e','f','g','h',
    'i','j','k','l','m','n','o','p','q','r','s',
    't','u','v','w','x','y','z',' '];

Since this is an uneven distribution, we need some way of representing each probability. In this method, we do this by “weighting” each value. We’ll store these weights in their own array:

var weights = [9, 2, 2, 4, 12, 2, 3, 2,
    9, 1, 1, 4, 2, 6, 8, 2, 1, 6, 4,
    6, 4, 2, 2, 1, 2, 1, 2];

In this case, the sum of all the weights is 102 (100 letters, 2 blanks). Therefore, the ratio of each weight to that sum is the frequency of the letter. (For example, there are 12 E tiles, so E’s frequency is 12/102, or about 11.8%.) That’s the key to this method. Basically, we do something like this:

function randomLetter() {
    var totalWeight = 0;

    for (var w in weights) {
        totalWeight += w;
    }

    var random = Math.floor(Math.random() * totalWeight);

    for (var i = 0; i < letters.length; i++) {
        random -= weights[i];

        if (random < 0) {
            return letters[i];
        }
    }
}

(Obviously, in a real game, we’d need something much more robust, with better error handling, etc., but this is enough to illustrate the point. Also, this isn’t normally how I’d write this function, but I’ve simplified for the same reason.)

The function works by picking a random number from 0 up to the sum of the weights. We then use that as an “index”, but not directly into the array. Instead, we count down from our chosen number, each time subtracting successive weights, until we go below zero, where we produce the corresponding letter.

Let’s say that our random number is 15. We go through the weights array, starting with 9. Subtracting 9 from 15 leaves 6, so we keep going, down by 2 to 4, then by 2 again to 2, then by 4 down to -2. That’s below zero, so that’s where we stop, returning 'd'.

This method isn’t just limited to choosing letters. You can use it anywhere you need a biased sample. Think of a game that has five different types of enemies, each with different chances of spawning. Set up a list of enemy types, another holding their appearance frequencies, and the same method will give you waves of bad guys.

(Note: I’m not claiming credit for any of this. It was all figured out a long time ago, and certainly not by me. I’m not even the first to write about it online. But it’s definitely something beginners should learn, and I hope to post more little “tutorial” articles like this in the coming months, because we were all young, once.)

Elan – Unicode and Emoji in your code

Everyone knows about Unicode. A massive set of thousands of characters, all ready to be used everywhere. Except in code. Sure, many programming languages allow accented letters (for example) in their names, and most support Unicode in comments or strings. But there are very few that harness the full power of the Universal Character Set. Most text editors are Unicode-aware, but we’re still forced to write :<= instead of ≤.

On the other side of the coin, the younger generation has fully embraced Unicode, whether they realize it or not. The latest revision of the Unicode standard added a number of “emoji”, emoticons and pictographs that are popular in messaging apps and elsewhere. With just a few symbols, emoji can convey a large amount of information, such as moods and activities. It’s even possible to write a story using only emoji. But you can’t write code with them.

Until now.

Introducing Elan

Elan (Emoji LANguage) is a new programming language I have created to merge these two disparate worlds. In Elan, almost every Unicode character, from emoji to Greek letters to Chinese ideographs to musical notes are usable. Every operation in the language can be shown with symbols, in addition to text. This functionality can give our code the same expressive power available to our smartphones. No longer do we have to be limited to ASCII. With Elan, the whole world is open to you.

In the rest of this post, I’ll give a brief description and tutorial on Elan. It’s always in development, though, and new features are on the horizon. The latest code is available at this Github repo, and I’ve also put up a live compiler.

A word about character support

I understand that not all systems, keyboards, and editors have support for entering the full range of Unicode characters. To help those unfortunate programmers that must use such hardware and software, Elan has an alternative input facility. Each symbol that can’t be typed on a US keyboard has an alternate form consisting of either a valid symbol or a few letters surrounded by colons. For example, the assignment operator ⬅ can also be written as :=:. The live compiler page lists all the symbols currently used in Elan with their corresponding alternates.

Variables

Variables are just names attached to values. In Elan, a name be a string of letters and numbers from any language, as long as you start with a letter. But Elan also allows single symbols as variable identifiers. (Specifically, they have to be symbols from outside the Basic Multilingual Plane of Unicode, but this is a current limitation of the compiler.)

This means that you can have variables named any of the following:
xyz ALongVariableNameWithMixedCase élan π こんにちわ 🌈

Variables can be assigned values. In many languages, this uses the equals sign (=), which causes confusion. With Elan’s support for Unicode, we can use ⬅ instead:

x ⬅ 1; twox ⬅ x * 2; こんにちわ ⬅ "hello"; TheAnswer ⬅ 42

As you can see, semicolons can be used to separate statements, but newlines work just as well.

Operators

Every language needs a set of operators. Elan includes the following:

  • The assignment operator ⬅, which you just saw.
  • The usual arithmetic operators: + - * / for addition, subtraction, multiplication, and division. The minus sign can also be used for negative numbers.
  • The modulo operator %, similar to many other languages. (This may get an alternate symbol in a later version.)
  • The power operator ^, that raises one value to the power of another. (e.g., 3^3 = 27)
  • Comparison operators < > = ≤ ≥ ≠. Note that, because we don’t use the equals sign for assignment, we can use it for comparison, just like in math class. (If you can’t type the “or equal” symbols, you can also use <= >= /=.)
  • Logical AND and OR, as & and |. (Logical NOT is an oversight that I’ll fix in a future version.)
  • Increment and decrement operators, ⬆ (:++:) and ⬇ (:–:), that take a variable and either add 1 or subtract 1.
  • A lot more that I’ll show you in the sections below.

Expressions can be as complicated as you need them to be. The order of operations should be familiar if you’ve used any other programming language, with multiplication coming before addition, etc. You can also use parentheses to make your intent clear.

Types of Values

Elan doesn’t have a static type system like Java or C++. But it does have the notion of different kinds of values: numbers, strings, booleans, lists, functions, objects, and null. Most of these should be obvious: numbers are numbers, strings are text inside quotes. Booleans can be either true (✔ or :t:) or false (✖ or :f:). Lists are like arrays in other languages, with multiple values accessed by number, while objects have values that are accessed by name. Functions are blocks of code that you can call. Null (🚫 or :null:) is simply the absence of a value.

Examples of values:

  • Numbers: 1 42 0.5 -17
  • Strings: "c" "string with spaces" "512"
  • Lists: {1,2,3} {"a","b"} {}
  • Booleans: :f: ✔
  • Null: 🚫

Functions and objects will be discussed below.

Conditional Expressions and Statements

The conditional expression is similar to other languages’ ternary operators. Given a condition and two options, it returns the first option if the condition is true, the second if it is false. The general format of a conditional is:

condition ? true-expression ! false-expression !

For example, x < 10 ? "less" : "more" produces the string "less" if x is less than 10, otherwise it is "more". (Of course, this doesn't handle the case where x equals 10, but that's for another time.)

If you don't need the "else" portion of the conditional, it still has to be present, but you can leave it blank: x < 0 ? x ⬅ 0!!.

A conditional statement is slightly different. It doesn't produce a value, so you can't assign it to a variable, but it can include multiple statements, including any of those shown below, such as loops or function definitions.

Functions

Functions allow us to do things. As such, they are an important part of Elan. To define a function, you use the ✏ (:def:) symbol. To its left are the parameters of the function, written as a list. To the right is a block of statements ended by the ◀ (:end:) symbol. You can have an empty parameter list with {}, but the body of the function must have at least one statement.

Inside the function, you can return a value by writing that value followed by the ↩ (:return:) symbol. (To return a boolean value, you can instead use the shortcut symbols 👍 (:yes:) and 👎 (:no:) for true and false.)

How you call a function depends on if it requires parameters. For a function that doesn't, you can use the :call: symbol, which you can write as either 📞 or 📱. To pass parameters to a function, you first use the ✉ (:with:) symbol with a list of parameters, then call the function as before. For example, 📞 f and ✉ {a,b} 📱 g.

Examples of functions include:

  • {n} ✏ n*2 ↩ ◀, which doubles any number passed to it.
  • {s} ✏ s.length ≥ 1 ? 👍 ! 👎 ! ◀, which returns true if a string is not empty.
  • {x,y} ✏ ✉ {(x^2 + y^2)} 📞 Math.sqrt ↩ ◀, the Pythagorean function.

By themselves functions you define have no names, but they can be assigned to variables to give them names. Doing this obviously allows you to call your own functions, and it also enables recursion, as in the classic Fibonacci function:

fib ⬅ {n} ✏ n < 2 ? 1 ↩ ! ✉ {n-1} 📞 fib + ✉ {n-2} 📞 fib ↩ ! ◀

Iteration and Loops

Elan has two kinds of looping. The first is iteration, which goes through a list and executes a block of code for each value in the list. (This is like a for loop in other languages.) For this, we can use the 🔁 (:iter:) symbol. On the left goes the list, which can also be a variable holding a list, while the code block goes to the right. Like with functions, the code block ends at the ◀ symbol. Inside the block, you'll most likely need the current value, and this can be obtained with the :i: symbol, which can be written as any of ☝, 👆, or 👇.

Examples:

  • {1,2,3,4,5} 🔁 ✉ {☝^2} 👍 console.log ◀ prints the squares of the numbers 1 to 5 in order.
  • values ✉ {☝} 👍 someFunction ◀ calls someFunction with each value in the values list. (This pattern is so common that it might get its own shortcut symbol in a future version.

Instead of iterating through a list, you can also set up a loop that will run until a condition is met, or until you specifically tell it to stop. These loops (called while loops in many languages) are defined by the :loop: symbol, either 🔀 or 🔃. This symbol is followed by the block to execute each time through the loop, and it can be preceded by a condition that will stop the loop.

Inside the loop, you can use the ◼ (:stop:) symbol to "break out of" the loop, ending it immediately. (This is the only way to stop a loop without a condition.) Also, the ⏩ (:ff: or :continue:) symbol skips the rest of the block, returning to the beginning of the loop.

For example, x ⬅ 1; x < 1000 🔀 ✉ {x} 👍 f; x ⬅ x*2 ◀ passes successive powers of 2 to the function f, stopping once it reaches 1000.

Choices

For choosing one path of code from among many, Elan has the choice statement, similar to other languages' switch. The general format of the choice statement is:

test-expression :switch: default-block :case: choice-1 :do: code-block :case: choice-2 :do: code-block etc.

The :switch: symbol can be written as 💡, while :case: can be either ☑ or 🔘.

Example: {x} ✏ x 💡 "many" ↩ ◀ ☑ 1 ➡ "one" ↩ ◀ ☑ 2 ➡ "two" ↩ ◀ ◀ defines a function that returns a word based on its parameter. The number 1 becomes "one", 2 becomes "two", and any other number returns the value "many".

Objects and Lists

Objects in the current version of Elan function in the same way as their Javascript equivalents. You create an object by using the :object: symbol, written as either ☺ or 📦. This is followed by a block containing assignments. Each assignment defines a property of the object.

Example: o ⬅ ☺ x ⬅ 42 ◀ creates a variable o which is an object. This object has a single property called x, which is set to 42.

(Note: In the current version of Elan, lists are essentially just objects whose properties are consecutive integers starting from 0. In other words, a list like {"a","b","c"} can be considered an object with properties 0, 1, and 2.)

You can access the properties of an object in one of two ways. First, the dot (.) works just like in Javascript, meaning that o.x on the above object will give us the value 42. The underscore (_) does exactly the same thing, but it is set to a much lower precedence, allowing you to use an expression like a_b-1.

New objects can be created with the 🔨 (:new:) symbol, which, at the moment, works just like the Javascript new. Example: a ⬅ 🔨 Array.

Error Handling

Currently, errors are handled using something like Javascript's exception handlers.

The general form for handling errors is:
:try: attempted-statements :catch: error-variable error-handler ◀

The :try: symbol can be 🔍 or 🔎, while :catch: is ✋. If an error occurs in the "try" block, then the "catch" block will be executed. The error variable is set when the error happens, and it is only used for the catch block.

You can throw errors from your own with the :throw: symbol, written as ⚾, 💣, or 💩. It is followed by an expression that will be passed as the error variable to any block that catches it. Example: 💩 "Something went wrong".

Conclusion

There are a lot more things that a language probably should do, but Elan is complete enough that you can play with it, tinker with it. I have plenty more ideas, but it's best to start small.

I hope you like what you've seen. If you have any comments, bug reports, or feature requests, leave them here or at the Github repo.

Have fun! 😎