Playing with Memoir

Last time, I talked a little about how I used Pandoc to create a paperback book. Well, since I wrote that, I’ve not only posted the thing, but I have a copy of my own. Seriously. That’s a strange feeling, as I wrote about on Patreon.

Anyway, I promised I’d talk about how I did it, so that’s what I’ll do. First off, we’ll look at Memoir, one of the greatest inventions in the history of computer-aided authorship.

Optional text

Memoir is a LaTeX class; essentially, it’s a software package that gives you a framework for creating beautiful books with less painstaking effort than you would expect. (Not none, mind you. If you don’t know what you’re doing—I can’t say I do—then it can be…unwieldy.)

It’s not perfect, and the documentation is lacking in some respects (the package’s author actively refuses to tell you how to do some things that upset his aesthetic sensibilities), but it’s far superior to anything you’d get out of a word processor. Oh, and it’s like code, too, which is great for logical, left-brain types like me.

So, let’s assume you know how to use LaTeX and include classes and all that, because this isn’t a tutorial. Instead, I’ll talk about what I did to beat this beast into shape.

First off, we’ve got the class options. Like most LaTeX packages, Memoir is customizable in the extreme. It’s not meant only for books; you can do a journal article with it, or a thesis, or just about anything that could appear in print. So it has to be ready for all those different printing formats. Want to make everything print only on one side of the page? You can do that. Multicolumn output, like in a newspaper? Sure, why not?

The list goes on, but I only need a few options. “Real” books are single-column and double-sided, so I’ll be using the appropriate class options, onecolumn and twosided. Books in English start on the right-hand page, so add in openright. But wait! Since most books use these options anyway, Memoir simply makes them the default, so I don’t have to do anything! (Now, if you’re making manga or something, you might need to use openleft instead, but that’s the exception, not the rule.)

Besides those, I only need to specify two other options. One is ebook, which sets the page to a nice 6″ x 9″—exactly the same as Amazon’s default paperback size. If you want something else, it can get…nontrivial, but let’s stick to the basics. Oh, and I want american, because I am one; this changes some of the typography rules, though I’ll confess I don’t know which ones.

Set it up

The remainder of the LaTeX “coding” is mostly a series of markup commands, which work a bit like HTML tags. The primary “content” ones are \frontmatter, \mainmatter, and \backmatter, which are common to Memoir and other packages; they tell the system where in the book you are. A preface, for instance, is in the front matter, and you can configure things so it gets its pages numbered in Roman numerals. Pretty much the usual, really, and not Memoir-specific.

For typography, some of the things I did include:

  • Changing margins. Amazon is finicky when it comes to these. It actually rejected my original design, because Memoir’s 0.5″ is apparently less than their 0.5″. So I’m using 0.75″ on the left and right for Before I Wake, and I suspect Nocturne will need something even bigger on the inside edge. Top and bottom get 1″ each, which seems comfortable.

  • Adding subtitle support. I don’t need this for either of the two novels I mentioned, but I might later on. Pandoc passes the subtitle part of its metadata through to LaTeX, but Memoir doesn’t support it. So I fixed that.

  • Creating a new title page. This was fun, for varying values of “fun”. Mostly, I just needed something functional. Then I had to do it again, to make the “half-title” page that professional books have.

  • Fixed headers and footers. This was mostly just configuration: page numbers in the outer corner of the header, author and title alternately in the middle, and footers left blank. Not too bad.

  • Changing the chapter style. Here’s where I almost gave up. By default, Pandoc tells LaTeX to create numbered chapters. Well, I did that myself. Rather than go back and change that (it would screw up the EPUB creation), I told Memoir to ignore the pre-made numbering completely. This is especially important when I get to Nocturne, because it has a prologue and epilogue. Having it put “Chapter 1: Prologue” would just be stupid.

  • Add blank pages. Now, you might be wondering about this one. Trust me, it’s for a good cause. Memoir is smart enough to add blank pages to make a chapter start on the right side (that openright thing I mentioned earlier), but it won’t do that at the end of the book, or if you go and manually make a title page, like I did. Oh, and if you’re doing a print book, remember that it ends on the left page.

The whole thing was almost a hundred lines of code, including the text for, e.g., the copyright and dedication pages. All in all, it took about three or four hours of work, but I really only have to do it once. Next time around, I just tweak a few values here and there, and that’s it. Automation. It’ll eventually take everybody’s job.

Coming up

So that’s enough to get something that looks like a book, but I’m still not done. Next up, you’ll get to see the bane of my existence: Pandoc filters. And then I’ll throw in a little bit about some interesting LaTeX packages I use, because I need Code posts. See you then!

Pandoc, LaTeX, and Memoir

A while back, I wrote about the “inner workings” of my writing. My stories are created using Markdown, which I run through a program called Pandoc to turn into EPUB format. (Then, to make Amazon happy, I send that through KindleGen, which spits out a MOBI file that can then go on the Kindle Store.) It works, and there’s a minimum of fuss. No fiddling with margins and page layout, no worrying about arcane or proprietary file formats, just a lot of text that already looks pretty much like a book.

Well, Amazon has a new thing for their KDP self-publishers: paperbacks. If you remember Createspace, it’s kinda like that, but integrated with the “main” Kindle Store. All you really have to do is upload a new format manuscript, and they’ll even give you an ISBN. (Note for non-US readers: my country seriously overcharges for ISBNs, so getting one for free is a big deal.) And the paper book shows up on Amazon as an option alongside the Kindle digital version. My brother already tried it with his book Angel’s Sin, and it seems to have worked.

So, of course, now I’m going to do the same with Before I Wake and the forthcoming Nocturne, as well as some of my future projects. To do this, however, I’ve had to delve deeper into the mechanics of my workflow.

The format issue

Amazon doesn’t like EPUBs. That’s well known. For digital books, they really, really want you to send them either a MOBI file, or something like HTML or a Word document. That’s most assuredly because of DRM. (It can’t be because they don’t know how to convert, since they give you a command-line tool to do so!) Be that as it may, I don’t really mind the last little step of running KindleGen to make an Amazon-friendly version; it’s easily automated, and I’ll still have the EPUB ready to go on Patreon or wherever.

With this new paperback option, however, there’s a problem: they don’t take MOBI, either! Nope, if you want to upload a manuscript for actual printing, your options are Word DOC/DOCX, plain HTML (possibly zipped with images and stylesheets), or “print-ready” PDF. That last is code for, “Do all the layout yourself, ’cause we ain’t touching it.”

Well, there’s the dilemma. Pandoc will happily output just about whatever format you like, but each of the options available has its downsides. Microsoft Word documents require (naturally) Microsoft Word, which isn’t really an option for a Linux user like myself. (The web app version of Office is also a nonstarter, for much the same reasons.) Zipped HTML is essentially an EPUB already, but then you have all the layout issues that come from shoving a “streaming” markup format like HTML into the “blocks” of a printed page. Fiddly bits like margins and headers and page numbers, and all with no usable previewer.

So what does that leave? Only one thing: PDF. And Pandoc can make a PDF, but not by itself. Fortunately, it knows someone who can help.

The type type

TeX (that’s really how it’s meant to be written in plain text) is the famous typesetting program originally developed by the equally famous Donald Knuth. I’ve used it many times before, on Linux and on Windows, and it works great for what it is: a “programmer’s” interface to text layout. Not a word processor, but a text processor.

TeX has been extended a few times over the past 40 or so years, and it has accrued an entire ecosystem of add-ons, bells and whistles, and documentation. If you’re willing to put in the work, you can get a seriously beautiful document. By default, it comes out in PostScript format, which is relatively arcane and not really useful to anyone. But far more common these days is its PDF option. Its print-ready PDF option.

I don’t mind writing a bit of code. I’d rather do that than play around in a word processor GUI, clicking at buttons and tweaking margins. Give me the linear word any day of the week. So I decided I’d try to use TeX (actually, the much simpler wrapper LaTeX, and be absolutely sure you capitalize that one right!) with Pandoc to make a printable PDF of one of my books.

Writing my memoir

The full story is going to play out over the next few weeks. I’ve been searching for new material for the “Code” posts here, and now I’ve found it: a deep look into what it takes for me, a very non-artistic writer experienced with programming in multiple languages and environments, to create something that looks like a book.

In the first of multiple upcoming posts, I’ll look at memoir, a wonderful LaTeX extension (“class”, as they’re called) used for creating books that truly look like they were designed by professionals. It’s not exactly plug-and-play, and I’ll gladly admit that I had to do a lot of work to beat it into shape, but I only had to do it once. Now, every book I write can use the same foundation, the same basic template.

After that, I’ll go back to Pandoc and show you the work I did to convince it to do what I wanted. I’ve never written a horror story before, but this might be the closest to it, from a programmer’s perspective. It was a coding nightmare, one I’m not sure I’m out of yet, but the end result is everything I need in a book, as you’ll see.

Practical Typescript: dice roller

In the last few “code” posts of Prose Poetry Code, there’s been one thing missing. One very important thing, and you might have noticed it. That’s right: there’s no code! I’ve been writing about generalities and theory and the like for a while now, but I’ve been neglecting the ugly innards. Part of that is because I haven’t been in much of a coding mood these past few months. Writing fiction was more interesting at the time. But I’ve had a few spare hours, and I really do want to get back into coding, so here goes.

A while back, I mentioned Typescript, a neat wrapper that sits on top of JavaScript and generally makes it palatable. And at the end of that post, I said I’d be playing around with the language. So here’s what I’ve got, a practical example of using Typescript. Well, dice rollers aren’t exactly practical—they’re a dime a dozen, and nobody really needs them—but you get the idea.

The code

I’ve posted a full ZIP of the code, including a config file, an HTML skeleton, and the JavaScript output. But here’s the Typescript code (dice.ts) for your perusal.

// Simple function simulating a single die roll
function roll(sides: number): number {
    return 1 + Math.floor(Math.random() * sides);
}

// Our "back-end" will roll a number of simulated dice,
// possibly adding a bonus or subtracting a penalty.
// It will return an object containing the rolls and their total.
// (This is really for no reason other than to show off interfaces.)
interface RollResult {
    rolls: number[],
    total: number
}

// Roll a number of dice, and return the results,
// using the object we defined above.
function rollMany(count: number, sides: number): RollResult {
    let rolls : number[] = [];

    for (let i = 0; i < count; i++) {
        rolls.push(roll(sides));
    }

    let result = {
        rolls: rolls,
        total: rolls.reduce((a: number, b: number) => a+b, 0)
    };

    return result;
}

// This is where the interactive portion of the script begins.
// It's really just basic DOM stuff. In a moment, we'll actually
// hook it all up to the page.
function diceRoller() {
    console.log("Rolling...");

    let countBox = document.getElementById("count") as HTMLInputElement;
    let sidesBox = document.getElementById("sides") as HTMLInputElement;
    let addsBox = document.getElementById("adds") as HTMLInputElement;
    let resultRollsText = document.getElementById("result");
    let resultTotalText = document.getElementById("total");

    let count = +countBox.value;
    let sides = +sidesBox.value;
    let adds = +addsBox.value;

    let result = rollMany(count, sides);
    let totalRoll = result.total + adds;

    resultRollsText.innerHTML = result.rolls.join(" ");
    resultTotalText.innerHTML = ""+ totalRoll;
}

// This clears out our results and options.
function clearAll() {
    console.log("Clearing...");

    // Note that this gives us an HTMLCollection...
    let boxes = document.getElementsByClassName("entry");

    // ...which can't be used like an array in for/of...
    for (let b in boxes) {
        // ...and contains only generic HTMLElements.
        (boxes[b] as HTMLInputElement).value = "";
    }

    let resultText = document.getElementById("result");
    let totalText = document.getElementById("total");
    resultText.innerHTML = "";
    totalText.innerHTML = "";
}

// Here's where we connect our functions to the page.
document.addEventListener("DOMContentLoaded", function () {
    let clearButton = document.getElementById("clear");
    let rollButton = document.getElementById("roll");

    clearButton.onclick = clearAll;
    rollButton.onclick = diceRoller;
});

Honestly, if you’ve seen JavaScript, you should have a pretty good idea of what’s going on here. We start with a helper function, roll, which does the dirty work of generating a number from 1 to N, exactly as if you rolled a die with N sides. (It’s not perfect, as JavaScript uses pseudorandom numbers, but it’s the best we can do.) If you take out the two type declarations, you wouldn’t be able to tell this was Typescript.

Next comes something that inarguably identifies our source as not normal JS: an interface. We don’t really need it for something this simple, but it doesn’t cost anything, and it’s a good check on our typing. Our RollResult interface simply defines an object “layout” that we’ll pass from our back-end rolling function to the front-end output. If we screw up, the compiler lets us know—that’s the whole point of strong typing.

After this is the rollMany function. It builds on the simple roll, calling it multiple times and storing each result in an array. This array, along with the sum of its contents, will become the returned object, matching our interface.

We’ll skip over the diceRoller function briefly, as it’s the grand finale of our little app, and I wanted to save it for last. Instead, we move to clearAll. It would be an unremarkable DOM manipulation function, if not for two quirks in Typescript. First, our simple HTML page defines a few entry boxes: one for the number of dice to roll, one for how many sides each die has, and one for a flat bonus or penalty to add to the total. These are <input> text elements, and they all have a CSS class of entry. Iterating over them (and only them) should be a piece of cake, right?

Not quite. See, the obvious way to do this, document.getElementsByClassName, returns an HTMLCollection, which isn’t quite the same thing as a JavaScript array. And Typescript, unless you tell it to spit out the latest and greatest ECMAScript standard (and I haven’t), won’t let you use the easier for..of loop. Instead, you have to use for..in, which iterates over the keys of a collection—for arrays, these are the indexes.

And that brings us to our second problem. See, our DOM collection is full of HTMLElements. These could be anything at all, and are thus usable as essentially nothing. In particular, we can’t change the value property that all input textboxes have, because not every HTML element has them. As any Java or C# programmer knows, we need an explicit cast into the more specific type of HTMLInputElement. That seems like needless drudgery, but it pays off in more complex applications, and we could always use jQuery or something instead. In a “real” setting, we probably would be.

The areas where we output our rolls and their total are simple divs. We don’t have to do any casting with them; we can just clear their innerHTML properties. And the last bit is not much more than your usual “let’s set up some event handlers” block.

That leaves diceRoller. First up are a few variables to hold the DOM elements: 3 text boxes and 2 output areas. Again, we have to do a cast on anything that’s an input element, but that should be old hat by now.

Following that, we make a few variables that hold our input values in number form. I used the idiomatic “unary plus” conversion from string to number here.

Next comes result, which holds (naturally) our resulting roll. I threw in a totalRoll variable to hold the total (including the “adds” roll bonus/penalty), but you don’t really need it; you can calculate that as you’re putting it into the output field. Speaking of which, that’s where we end: joining the result array into a space-separated string (you could use commas or whatever), then putting that and the total into their proper places.

Conclusion

So Typescript isn’t that hard. Counting comments and blank lines, this little thing weighs in at about 80 lines. The resulting JavaScript is less than 50. The difference comes from the strong typing, an interface definition that is purely for the benefit of the Typescript compiler and the programmer, and a few other odds and ends that contribute nothing physical to the finished product.

It may seem silly. We’re writing more code, locking ourselves into the type system, and we’re not really getting much out of it. Sure, we’ve got protection from typos. If I’d misspelled the name of one of the RollResult fields, I’d get an error telling me where I went wrong, but that’s about it.

For something this simple, even that’s enough. I get that error when compiling. I don’t end up with a blank screen or unresponsive page and no indication as to why. JavaScript’s dynamic nature is great, but it’s also terrible. In coding, unlike in real life, there’s such a thing as too much freedom.

Now, if you like, feel free to take this tiny app and improve on it. The HTML, for instance, expects a stylesheet; you could make one. Add in some bells and whistles. Throw up a few preset buttons. Store the last 100 rolls in DOM local storage. Most of all, have fun with it.

First glance: Scala.js

I’ve learned and used a lot of different programming languages. If I had to pick one to take as the moneymaker, it’d probably be JavaScript right now. But JavaScript is pretty awful. It wouldn’t be my first choice if money was no object. That would likely be C++, but there’s one other contender, one language I liked from the first time I saw it. That language is Scala.

Scala has its problems, of course. It’s closely tied to the Java ecosystem, which…isn’t the greatest. Sure, it’s got a huge selection of libraries and frameworks for whatever you want to do, but the Java language itself is atrocious, the JVM has a well-deserved reputation for bloat, and it’s practically banned on the desktop. (And you run the risk of getting sued if you use it on mobile, as Google discovered.) So Scala isn’t exactly a popular option, for very good reason.

But it doesn’t have to be that way. Plenty of languages have been turned into tools for web development by the clever trick of compiling them into JavaScript. For Scala, that would be the job of Scala.js. It’s a relative newcomer (version 0.6.14 at the time of this writing), but it’s already looking good.

What is it?

Scala.js is, simply enough, a tool for turning Scala code into JavaScript. In that, it’s not much different from “transpilers” like TypeScript or Babel. Only the source language isn’t something vaguely reminiscent of the JS it will become. No, you write in Scala, so you have all the expressive power of that language.

I like Scala. It’s one of the few languages I’ve seen that accomplishes the feat of getting me interested in functional-style programming. I absolutely hate the “pure functional” approach of, e.g., Haskell, where you can’t actually do anything, because all those nasty “impurities” like, say, I/O, are restricted, and variables can’t, you know, vary. (Yeah, you can use monads, but they’re limited in how they interact with the rest of your code. That’s kinda the point.) Scala, instead, makes FP style encouraged, but not required. That’s much better.

On top of that, Scala is just a better Java than Java. It’s got most of what Gosling took out as “too hard”, like operator overloading. It’s got pattern matching. Lambdas exist, and they’re easy to write. Scala is what Java programmers wish they could use. Putting it in the browser? Sure.

The good

So we’ve got a better language, and that’s where most of the good comes from. Like what, you may ask? How about strong typing? Sure, you can get that in TypeScript, but Scala uses it to its full extent. The functional style is front and center, and it’s not hidden behind bad syntax like in JS. OOP is class-based, not prototype-based, and you don’t have to worry about browser compatibility. And then there are a lot of things Scala has that just aren’t feasible in JS, like case classes and traits.

Outside of the language proper, you’ve got a good HTML templating system, Scalatags, that has full type safety. And it’s not an addon, not in the JS sense. No, you’re working with something that’s based on Java, remember? You’ve got Java-derived build tools and libraries.

I could go on, but the main advantages of Scala.js come from the simple fact that you’re using Scala in place of JavaScript. Everything else great about it follows naturally from that. If you like Scala, that should be enough to give it a shot. If you hate JavaScript, same deal.

The bad

But it’s not all good. That same Java heritage is the greatest weakness of Scala.js. You’re using Java tools, a Java-based API, JVM-centric constructs. Not a fan of Java? Tough. You really can’t get away from it with Scala. (Really, though, this isn’t that much different from that thing Google did a few years ago. JWT? I can’t remember, and I’m too lazy to look it up.)

A worse shortcoming is the number of limitations Scala.js has on what Scala code you can use. Mostly, this makes sense—you can’t use parallel programming in the inherently single-threaded environment of the browser. You also can’t use any Java libraries; your code, and all the code you import, has to be pure Scala, with a very few exceptions. And then there are a few nits to pick, cases where the developers of Scala.js couldn’t figure out a way to make the construct work in JS. Reflection is one of these: you can’t use it at all, and you can’t use anything that uses it. Macros (the recommended replacement) are a poor substitute for reflection, just as templates are in C++.

Lastly, you’ve got the build environment. If you’ve used any sort of JavaScript tools made this decade, you’re probably familiar with Node, Grunt, Gulp, and the like. Scala.js inherits Scala’s sbt, which is yet another thing to learn. And IDEs don’t seem to like it very much; I’ve never managed to get an sbt-based project talking to an IDE, and Eclipse, Netbeans, and IDEA all laughed at my attempts to use sbt for their projects. Since Scala, like Java, really wants an IDE, this can be a problem. (Maybe they’ve fixed things since then, though.)

The verdict

Although the “bad” section outweighs the “good” in quantity, don’t let that make you think I don’t like Scala.js. I like the idea of it. I can even like the implementation. File sizes are a bit on the hefty side, but they’re working on that. Some of the API limitations can be changed. The thing’s still in beta, remember.

On the whole, if you can live with the restrictions it places on you, Scala.js looks to be a very interesting way of creating web apps without writing raw JavaScript. I know I’m keeping an eye on it.

Playing with TypeScript

JavaScript sucks, and we all know it. But it’s the lingua franca of the web (not like WebAssembly is ever taking off), so we have to learn to deal with it. That doesn’t, however, mean we have to write it. Oh, no. There’s a whole subculture of languages that compile into JavaScript, meaning you can write your code in something that looks sane (to you) without worrying about the end result being readable.

This trend really started a few years ago with CoffeeScript, which did an admirable job of making JS look like Ruby. From there, similar projects spun off, like Coco and LiveScript, which were far better in that they didn’t have anything to do with Ruby. And then Microsoft got involved. That’s where TypeScript comes from. Yes, that Microsoft.

But it’s not as bad as it sounds. The language and compiler use the Apache license, and that gives us a bit of protection from the worst of corporate silliness. And that means it might be interesting to look at TypeScript for what it is: an attempt at making a better JavaScript than JavaScript.

Up and running

TypeScript fits into the usual Node-hipster-modern ecosystem. You can install it through npm, and it works with most of the other big JavaScript tools like testing systems. Some of the big “web app” frameworks even use it by default now, meaning that I’m a little behind the times. Oh, and you can also use it with Visual Studio, but I can’t, because that’s Windows-only. (And VS Code is not the same.) But you don’t have to: the homepage even has a link to TypeScript support for the best text editor out there. I’ll leave it to you to decide which one that is, but I’ll tell you that it’s the same one I’m using to write this post.

My type

So you get TypeScript installed, and you’ve got a nifty little compiler that spits out perfectly fashionable JavaScript, but what about the source language? What’s so special about it? In a word: types. (You’ll note that this word makes up half the language’s name.)

Essentially, TypeScript is nothing more than JavaScript with strong typing. That may not seem like much, but it’s actually a huge change that alters the whole way you write code. Variables are typed, functions are typed, objects are typed. To be fair, they all are in JavaScript, too, but that language doesn’t do anything with those types. TypeScript is made to use them. By itself, that’s almost enough to overcome my instinctive hatred of all things Microsoft. Almost.

On top of the bare bones JavaScript gives us, we’ve got tuples, enums, generics (C#-like, so not the best we could get, but far better than what JS offers), and an any type that lets us ignore the type system when necessary. If you wanted to, you could just about get away with writing regular JavaScript, tagging everything as any, and running it through the TypeScript compiler. But why would you do that? You’re supposed to be using a better language, right? So use it! Besides, like any decent strongly-typed language, we can use type inference to save us most of the trouble of specifying what something should be.

The next level

Just having the added safety of stronger typing already gives TypeScript a leg up on its parent language. But if it didn’t offer anything else, I’d tell you not to bother. Fortunately for this post, it has a lot more.

JS is currently in a state of flux. Most (but not all) browsers support ES5, some let you use a lot of the new ES6 features, and the language standard itself has transitioned to the ridiculous rapid-release model that’s all the rage today, so it’s looking like we’ll soon be back in the bad old days of “Best viewed in {browser X}”. Polyfills can only take us so far, you see. For coders writing raw JavaScript, it’s a serious problem making something maximally compatible, and most just throw their hands up and say, “Just use Chrome like everyone else.”

That’s another case where I’ll give the MS team credit. TypeScript takes a lot of the newer JS additions and lets you use them now, just like Babel or other “transpilers”. You can declare your variables with let instead of var, and the compiler will do the right thing. You get the new class syntax for free, which is great if you never could wrap your head around prototype OOP. Generators aren’t even in JS yet, so TypeScript even looks a bit like the future in some cases.

Reservations

If there’s anything wrong with TypeScript, it’s not the kind of thing that shows up in a cursory inspection of the documentation. No, the main problems are twofold. One, it’s a project started by Microsoft, a company with a history of bad blood towards the open source community. The Apache license helps in that regard, though I don’t think it can ever completely alleviate some people’s fears.

Second, TypeScript has been around for a while now, but it’s only recently been picking up steam. Angular and React both like it a lot, as do some indie game engines like Phaser. But the JS community is fad-driven, and this new acceptance could be an indication of that. If TypeScript is simply “the next big thing”, then interest will fade once some other shiny thing catches the eyes of the hipsters. We can’t prevent that, but we can do our best to ignore it.

Will TypeScript become the future of web development? I can’t say. It’s definitely one option for the present, though. And it’s a pretty good option, from what I’ve seen. I think I’ll play around with it some more. Who knows? Maybe I’ll show you what I’ve made.

Programming in 2016: game development

I tried to make a game this year. My body failed me. But I’ve been keeping up with the news in the world of game development, and 2016 has been exciting, if a bit frustrating.

Unity

Unity’s still the big kahuna for indie development. But they’ve gone to that same “rapid release” model that everyone else has, the same one that has all but ruined Firefox, Windows, and so many other projects. On top of that, they switched to a subscription model. Rather, they switched to a subscription-only model.

Yes, that’s right. You can only rent the Unity engine on a monthly basis now. It’s still free for tiny devs, but it actually costs more now for everybody else. Sure, there’s the new Plus tier (something like $40 a month, I think), but it doesn’t give you much over the Free version. By the time you need it, you can probably afford the full subscription.

On the technical side, they’re making progress towards Vulkan support, and there are rumblings about actually upgrading their version of C# to something approaching modern. That’s probably thanks to the .NET Core open-sourcing I mentioned last week, but I don’t care what the reasoning is. Any upgrade is welcome here.

The other rumor is that they might switch to C++. I…don’t know about that one. On the one hand, I have to say, “Yes, please!” Modern C++ is just as good as C# in almost every way. In many, it’s better. However, what does this do to that huge body of C# Unity code? If there’s a compatibility layer, then you’ve got inefficiencies. If they simply include the “old” engine, they’ve only made more work for themselves. And then you have JavaScript, which is still (mostly) a supported language for Unity coding. How would it fit in to a C++ future?

Godot

Godot is still my favorite 2D engine. It’s free, the source is open, and it’s very easy to use. 3D is a known problem, but that doesn’t bother me much; I’m not capable of making a 3D game anyway.

Well, Godot made their big announcement back in the summer, with the release of version 2.1. It’s not really revolutionary, but it sets the stage for greater things. Time will tell if those come to pass, but I think they will. With 2.2, we’re supposed to get a better renderer and possibly C# support. The big 3.0 might even add Vulkan to the mix, not that it helps me. And the Asset Library, well, it can only get bigger, right?

The main problem for Godot has been its documentation, and that’s much improved over this time last year. There’s a growing body of tutorials out there, too. I don’t think the engine has reached critical mass yet, but I also don’t think it has peaked.

Maybe—if I don’t get sick the day after I announce it—I’ll try another “game in a month” thing. If I do, it’ll be in Godot.

Lots of little ones

I didn’t do much in the way of development in 2016. I didn’t look at Unreal in anything other than passing, for example. But I’ve kept an eye on happenings in the game dev world, and here are some quick thoughts on other engines out there:

  • Unreal is, like the C++ it’s written in, solid and relatively unexciting. That’s what makes it exciting.
  • Superpowers might be a nice little JavaScript platform, but it’s got this horrible bug that makes all the dropdown boxes turn solid black. Makes it hard to use, you know?
  • Clickteam Fusion may or may not be getting bigger in 2017. They’re working on their version 3 release, and it might be cross-platform. Stay tuned for more on that front.
  • Amazon put out their Lumberyard (a fork of CryEngine). It’s free, as long as you’re willing to use their cloud services, but the real cost is in the machine you need to run the environment.
  • CryEngine itself is…strange. They’ve put out source code, but it’s not open. In fact, reading the license, t’s almost impossible to find a game you could even make! Maybe they’ll fix that, but I wouldn’t hold my breath.
  • The Atomic Game Engine looked like a promising release a few months ago, but it seems to be dead. The developers haven’t put out any news since May, and the forums were shut down in favor of Facebook. Sounds like they don’t want new users to me.
  • Finally, RPG Maker has a new version. It’s finally becoming something other than Windows-only, and the coding part has followed the hipster crowd from Ruby to JavaScript. In my opinion, that can only be a good thing.

I could go on, but I’m running out of year, so I’ll stop. Let’s just say this: 2016 was a good year for an independent game developer. 2017 will be even better. You’ve got a massive selection of engines at your disposal, from solid open-source offerings to AAA beasts. Maybe next year will be when we finally solve the asset problem. We’re getting there, slowly but surely.

Programming in 2016: languages

It’s nearing the end of another year, and that’s the cue for sites the world over to start looking back on the past twelve months. Due to a lack of creative impulse in the code sphere, I’ll be doing the same. So let’s see how the science and art of programming fared in 2016, starting with the advances and changes in programming languages.

JavaScript

JavaScript might not be the biggest language out there, but it’s certainly the one most people have experienced in some form, so it makes sense to start here. And JavaScript has certainly seen some improvements this year. For one thing, it’s got a new version, as the standards guys have moved to the same silly release model that gave us modern Firefox and Chrome. The only things added to ES2016 (what should have been ES7) are an exponent operator and a single new Array method, includes(). Absolutely nothing to get excited over, and the “big” changes, like async, are still in the future…if they’re ever put in at all.

On the other hand, the environment for JavaScript is getting better all the time. You can count on full ES5 support now, and you shouldn’t have too much trouble using ES6 features. Maybe you’ll need a polyfill for those, but that’s a temporary solution. And the one good thing about Windows 10 is Edge, if only because it’s the end of the “old” Internet Explorer and that browser’s awful support for, well, anything.

Outside the browser, Node keeps chugging along. They’ve had some problems there (such as the leftpad debacle), but they’ve got that whole Node/IO.js fork thing worked out. From a political standpoint, it’s not optimal, but the codebase is solid enough. The ecosystem is growing, too, as almost everybody is using some sort of Node/Webkit construction, from Visual Studio Code to Atom to Vivaldi.

As usual, JavaScript’s future looks much brighter than its past. It’s still straining at its own boundaries, but it’s growing. It’s becoming a better language. The two main platforms (browser and Node) have improved by leaps and bounds, and now they’ve become mature. In the next year, they’ll only get better.

C++

C++ had a big year in 2016, but it was all behind the scenes. The real test will come next year, when the C++17 standard comes out, but we already know what it’s going to have in it. What might that be? More of everything, really. I’ve already written about some of the more interesting bits in a three-part series back in August. (Part 1, Part 2, Part 3)

So 2017 looks like it’ll be fun, but what about now? “Modern” C++ is finally getting widespread support, so that’s good. Even those of you stuck on five-year cycles should now be ready for that, and C++14 was an incremental upgrade. On the platform side, it’s C++. The platform is your computer, or your phone, or your refrigerator. It’s the same platform it’s always been. That’s why you would use it over any other language.

C

Microsoft has been doing their schizophrenic love-hate thing with open source for a while now, but 2016 saw the biggest payoff of the “love” side. The open release of .NET Core happened this summer, complete with support for platforms other than Windows. Considering how hard it can be to get MS to even acknowledge such a thing, that’s practically a miracle.

It’s also the beginning of the end for Mono—the third E stands for Extinguish, remember—but that’s not as bad as it sounds. Mono still has its uses, mostly for older .NET code and the things that the MS offering won’t support. Oh, and they also bought Xamarin, whose main reason for existing seemed to be letting you write C# code for mobile devices. But if even some of .NET is open-source, we don’t need that anymore. The community can do it.

C# remains a Microsoft language, though. The MIT license won’t change that. It’ll definitely give non-Windows developers some peace of mind, but you always have to remember who holds the cards here.

Java

Java’s a funny thing. On the one hand, it’s not that bad a platform. On the other, it’s owned by Oracle. They’re still playing the part of Sisyphus, rolling a ball of lawsuits up the hill, only for the courts to send them right back down. But unlike in mythology, there’s the possibility that they might win.

If they do, it’s bad news for everyone, but most of all for Java developers. Who would want to use a language owned by a sue-happy corporation? (And now the C# fanboys can say, “There’s another thing Java ripped off!” Well, okay, at least MS hasn’t sued anybody using C#. Yet.)

But if you can put that out of your mind, Java’s not bad. It’s awful from a security standpoint, and the language sucks, and—well, you get the idea. I still believe it has some upside, though. Scala—or Clojure, if that’s the way you roll—makes using the JVM bearable. Android requires Java coding, even if they desperately want to call it something else.

So Java’s 2016 has been mostly dismal, but look on the bright side: Java 9 is coming soon. And one proposal on the table is “ahead-of-time compilation”. For Java. Just think about that for one second. Maybe I’m misunderstanding, but wasn’t that the exact thing Java was designed to prevent?

The rest

Other languages have had an interesting year, but it’s too much work to list them all. Besides, I haven’t used that many of them. But here’s a few of the highlights:

  • Rust is growing. It’s almost to the point of being usable! (The rewrite of Firefox into Rust will come before that point, however.)
  • Go is still bad, but at least Google isn’t trying to push it (or Dart) on Android devs anymore. I’ll call that an improvement.
  • Python really seems to be disappearing. It’s had some minor releases, but nothing amazing. That’s a good thing, but the glaring flaws of Python 3 haven’t gone away.
  • PHP remains the bad joke of the programming world, but maybe PHP 7 can fix that. Because PHP 6 did so well.
  • Perl 6…exists! Technically, that happened last Christmas, but it’s close enough. You can’t complain about a once-in-a-lifetime occurrence.

We’ll round out the year next week by looking at another area of development: games.

Wrapping text by code

Very many games require displaying significant amounts of text on the screen. Whether dialogue, signs, backstory “data log” entries, or lists of quests and objectives, text is absolutely a necessity. And we’re not talking about just a word here and there. No, this is serious text, entire sentences at least. More specifically, I want to look at the long strings of text that are commonly shown in dialog boxes or other GUI/HUD elements, not something that might be baked into a texture.

With single words, it’s not hard at all: you just throw the word on the screen at the right position. What game engine doesn’t let you do that? Even my failed attempt at a simple 2D engine a few years ago could manage that much!

It’s when you need to display longer snippets that things get hairy. Your screen is only so wide, so, like any word processor, you have to know when to wrap the text. Now, many engines do this for you, but you may not be using one of those, or you may be writing your own. So this post is designed to give you an idea of what lies ahead, and to provide a starting point for your own efforts.

Ground rules

First, let’s get this out of the way: I’m using a bit of pseudocode here. Actually, it’s JavaScript, but with a lot of references left undefined. Hopefully, you can convert that into your language and platform of choice.

Next, I’m going to assume you’re working in English or another language that uses an alphabetic script. Something like Chinese or Japanese will have its own rules regarding how long text is broken up, and I don’t entirely understand all of those. The same goes for Arabic and other RTL scripts, but they’re much closer to American and European usage, so some of the concepts described here might carry over.

We’ll also ignore hyphenation in this post. It’s a hideously complex topic, first of all, and nobody can quite agree on which words should (or even can) be hyphenated at the end of a line. Besides, what was the last game you saw that used printed-style hyphenation? In the same vein, we’ll be ignoring non-breaking spaces, dashes, and the like.

The last assumption I’m going to make is that we’re working with Unicode strings. That shouldn’t matter too much, though. We won’t be delving into individual bytes—if you do that with Unicode, you’ve gone wrong somewhere, or you’re at a far lower level this post.

Words

On the screen, our string will be divided into lines. Thus, our job is clear: take a string of text, divide it into lines so that no line is longer than the width of our dialog box. Simple, right? So let’s see how we can go about that.

English text is divided first into sentences. These sentences are then divided into words, sometimes with added punctuation. Each word is separated from its neighbors by blank space. It’s that space that lets us wrap our text.

For a first approximation, let’s take the example of fixed-width text. This was common in older games, and it’s still used for a retro feel. (And programmers demand it for editing code.) It’s “fixed” width because each letter, each bit of punctuation, and each space will have the same width. That width can be different from one font to another, but every character in a fixed-width string, even the blank ones, will be as wide as any other.

As we won’t be worrying about hyphenation, every line will have a whole number of words. Thus, our first task is to get those words out of our string:

// Split a string of text into a list of words
function splitWords(str) {
    return str.split(' ');
}

Basically, we’ve split our string up into words, and removed the spaces in the process. Don’t worry about that. We’ll put them back in a moment. First, we need to go on a little digression.

Building with boxes

Do you remember magnetic poetry? Those little kits you could buy at the bookstore (back when bookstores were a thing)? They came with a set of pre-printed words and a bunch of magnetic holders. Some of them even let you cut your own magnets, then slide the bits of laminated paper containing words into them. However you did it, you could then assemble the words into a short, tweet-level string that you could stick to your fridge. Longer words required longer magnets, while something like “a” or “I” took up almost no space at all. If you used one of the magnetic poetry kits that came with a little board, you had to do a bit of shuffling—and maybe some editing—to get the words to fit on a line.

That’s what we’re doing here. We’re taking the words of our text, and we’re assembling them on the screen in exactly the same way. Like the board, the screen is only so wide, so it can accommodate only so much. Any more, and we have to move to the next line. So wrapping text can be thought of like this. (It’s also the beginnings of computerized typesetting. We’re not really doing anything different from TeX; it just does the job a whole lot better.)

So we need to pack our words onto lines of no more than a given width. One way to do that is incrementally, by adding words (and spaces between them) until we run out of space, then moving on to the next line.

// Build lines of fixed-width text from a string
function buildTextLines(str, screenWidth, fontWidth) {
    let allWords = splitWords(str);
    let lines = [];

    let currentLine = "";
    let currentLineWidth = 0;
    let currentWord = "";

    while (allWords.length > 0) {
        currentWord = allWords.shift();

        let currentWordWidth = currentWord.length * fontWidth;

        if (currentLineWidth + currentWordWidth > screenWidth) {
            // Too long; wrap text here
            // Save the line we just finished, and start a new one
            lines.push(currentLine);
            currentLine = "";
            currentLineWidth = 0;
        }

        currentLine += currentWord;
        currentLineWidth += currentWord.length * fontWidth

        // Try to insert a space, unless we're at the edge of the box
        // (width of space is the same as anything else)
        if (!(currentLineWidth + fontWidth > screenWidth)) {
            currentLine += " ";
            currentLineWidth += fontWidth;
        }
    }

    return lines;
}

For fixed width, that’s really all you have to do. You can still treat the text as a sequence of characters that you’re transforming into lines full of space-delimited words. As a bonus, punctuation carries over effortlessly.

Varying the width

The real fun begins when you add in variable width. In printed text, an “m” is bigger than an “n”, which is wider than an “i”. Spaces tend to be fairly narrow, but they can also stretch. That’s how justified text works: extra bits of space are added here and there, where you likely wouldn’t even notice them, so that the far ends of the lines don’t look “ragged”. (That, by the way, is a technical term.) Two words might have an extra pixel or two of space, just to round out the line.

Doing that with simple strings is impossible, because I don’t know of any programming language having a basic string type that accounts for proportional fonts. (Well, TeX does, because that’s its job. That’s beside the point.) So we have to leave the world of strings and move onward. To display variable-width text, we have to start looking at baking it into screen images.

Since every engine and renderer is different, it’s a lot harder to give a code example for this one. Instead, I’ll lay out the steps you’ll need to take to make this “box” model work. (Note that this phrase wasn’t chosen by accident. HTML layout using CSS, as in a browser, uses a similar box-based display technique.)

  1. Using whatever capabilities for font metrics you have, find the minimum width of a space. This will probably be the width of the space character (0x20).

  2. Using this as a guide, break the text into lines—again based on the font metrics—but keep the words for each line in a list. Don’t combine them into a string just yet. Or at all, for that matter.

  3. For each line, calculate how much “stretch” space you’ll need to fill out the width. This will be the extra space left over after adding each word and the minimum-width spaces between them.

  4. Divide the stretch space equally among the inter-word spaces. This is aesthetically pleasing, and it ensures that you’ll have fully justified text.

  5. Render each word into the sprite or object you’re using as a container, putting these stretched spaces between them. You can do this a line at a time or all at once, but the first option may work better for text that can be scrolled.

In effect, this is a programmatic version of the “magnetic poetry” concept, and it’s not too much different from how browsers and typesetting engines actually work. Of course, they are much more sophisticated. So are many game engines. Those designed and intended for displaying text will have much of this work done for you. If you like reading code, dive into something like RenPy.

I hope this brief look whets your appetite. Text-based or text-heavy games require a bit more work for both the developer and the player, but they can be rewarding on a level a fully voiced game could never reach. They are, in a sense, like reading a story, but with a bit of extra visual aid. And games are primarily a visual medium.

RPG Town: layout

I love the retro look in games, that 16-bit pixel-art style that has, thankfully, become common once again in indie titles. A Link to the Past is my favorite Zelda game, and I’ve been playing Stardew Valley far too long over the past few months. Maybe it’s the nostalgia talking, but I truly enjoy this style.

One of the main draws of retro graphics is the way an area can be laid out using a tilemap. There are far too many tilemap tutorials out there, and this won’t be one of them. Instead, I’d just like to share a little thing I started recently. I call it RPG Town.

The town

RPG Town is a little village that would be at home in any RPG, action-adventure, or roguelike game that uses a 16-bit graphical style. It’s definitely a pixel-art place, and I’m using a free sprite set I found linked on the excellent OpenGameArt.org to visualize it.

The backstory, such as it is, is deliberately vague. RPG Town can be a home base for a player character, or a little stop along the way, or just anything. It’s home to a couple dozen people, and it’s got that typical over-urbanism common to video game settlements. No real farms here, but you can imagine they’re right off the map. It’s also a coastal place, situated on the mouth of a small river. Thus, fishing is likely a big deal there. There’s an offshore island not too far away; it’s claimed by RPG Town, but the people don’t really use it for much. (Maybe it holds the ruins of a lighthouse, or something more sinister.) Other than that, it’s just your average, quaint little town, set amid fertile plains and dense woodlands, ready for an adventure to come its way.

The making of

Really, RPG Town is an excuse for me to play with Tiled. I’ve never used it for much, but I want to. It’s a wonderful program, great for creating tilemaps of any kind, but especially those for retro-style games. Plus, it’s free, and it runs everywhere, so you don’t have to worry about any of that. Get it. It’s more than worth your time.

Anyway, after starting a new map and setting up the tile set I’m using. My first step was to configure the terrain. The Tiled manual talks about doing this; it’s a huge help, if your tile set is made for it.

Once I had the preliminaries out of the way, it was time to start making a map. So that’s what I did. First, I filled my whole canvas (128×64, by the way, with the tiles 16 pixels square) with a plain grass tile, then “carved out” the water areas. I had already decided RPG Town’s location, so all I had to do was draw the water. Tiled took care of the tile transitions seamlessly, and I ended up with this:

rpgtown-terrain

Tilemaps are best built in layers, from the base (the terrain) up to the more complex parts like houses or signs. Following that philosophy, next come the roads. RPG Town isn’t a big port, but it’s a port, so it’s only natural that it would be a stopping point for a road or two. Those I made as wide cobblestone paths, meeting in the middle in a tiny town square. There are also a few branching side streets, and some stuff that won’t quite make sense yet. Oh, and I added a spot of sandy beach, because who doesn’t like that?

rpgtown-roads

Now that I know where all the roads are running to, it’s time to give the people of RPG Town some places to live and work. I’m not ready to actually plop down buildings yet, so I’ve limited myself at this early stage to simply staking them out. On this last picture, you can see the outlines of where buildings would go. Later on, I’ll build them, but this is enough for now.

rpgtown-layout

To be continued?

I’ll keep playing with RPG Town in my spare time. If I do anything interesting with it, I’ll post about it. One day, there might even be a few virtual people living there.

Gamification and coding

One of the big trends this decade has been gamification. By turning everyday tasks into games, the theory goes, a person’s natural competitive streak will be triggered, causing that person to put more effort into accomplishing the task, and accomplishing it well. Companies do it with their employees, web forums have begun to implement it for their users and moderators, and so on.

It was only a matter of time before someone brought it to programming. Actually, that happened years ago. Coding competitions have been around a long time, but I saw a link to CodinGame the other day, and that got me thinking.

Games people play

The idea behind gamification isn’t hard to understand. Humans are competitive by nature. We like to win, to be better than the rest. And we really like it when there’s an objective measure of just how much better we are. The Olympics weren’t that long ago, and how many of you remember watching in anticipation, staring at the world records in the corner of the screen? How often were you on the edge of your seat, waiting for the judges’ scores to pop up?

It’s not just sports, though. Most games have some element of scoring, some record of accomplishment. In Monopoly, for example, you’ve got an obvious one: the amount of money in front of you. (Unless you’re playing one of those newer versions with the credit cards, in which case, why?) In video games, tables of high scores have existed for decades, especially in arcade games. And we wanted to set those high scores so bad, because that brought a small measure of fame. Everyone could see that MHP (or whatever) was the best at that game, at least until somebody posted a better score.

Today, when we play over the Internet instead of in public, we have online leaderboards instead, but the principle is the same. It drives us to improve, to reach for that top spot. (Story time: While my brother was working at Amazon a few years ago, I spent about three hours on his XBox, playing Forza 4 and trying to set a record in one of its weekly challenges, a quarter-mile drag race. It was some of the most fun and satisfying gameplay I’ve ever had, even though I never got higher than about 26th.)

Achievements are another popular aspect of modern games, and they work the same way. As Pokemon taught us: you gotta catch ’em all! And that’s how it is with achievements. We see that list, those empty spots, and we have goals. We have something to strive for.

That’s what gamification is. It’s the transfer of that competitive urge, that desire to complete a set or simply win, to mundane work. By adding points and trophies and collectibles, we transform drudgery into entertainment. Whether at school, on the job, or in a social setting, these artificial goals effectively make us believe we’re playing a game. Games are fun, right? So if we make the boring stuff into a game, then it magically becomes fun! Well, maybe. It doesn’t always work, and it’s not for everybody.

Games without frontiers

In theory, almost anything can be gamified, but we’re talking about programming here, so let’s stick to that. How can we make writing computer programs into a game? There are a few obvious answers. One is to make a game about coding, as with else heart.break() or TIS-100. That works, but it’s very much a niche. Another option is adding subtle programming abilities into an otherwise unrelated game. Minecraft has redstone, for example, which allows you to build logic gates, the building blocks of computing, while Final Fantasy XII gave players a bit of power to program their AI-controlled party members. In those cases, however, the programming part is not the focus. It’s an aside, sometimes an unwelcome one.

True gamification of coding, as CodinGame does, is different. It’s more of a series of programming challenges, each with objective measures of success and prowess. Code has to be correct, first and foremost. It has to do what it was designed to do. It’s also good if it’s fast; given two identically correct programs, the faster one is usually considered better. Space is another factor—smaller is generally better—and you can come up with other metrics. (“Lines of code”, by the way, is not a very good one.)

Once you have a way of measuring the “score”, you’re halfway to making a game of it. Post the scores publicly, and watch as coders eagerly train at their craft for no other reason than to move up a spot or two. It’s almost like magic!

Can it work, though? I don’t know. It works for a lot of people, but not everyone. We’re not all the same. Some of us don’t have that competitive streak, or we don’t act on it. For them, gamification is a waste of time. But for the rest of the populace, it can create a reason to learn, a reason to want to learn. That’s something the world could use a lot more of, so I say it’s worth a shot.