I recently converted Subtex from D to C#. With the same amount of optimization, the C# version was thirty times slower. Let’s look at why.
I implemented Silvermirror, a website mirroring tool. It’s got some interesting aspects that make it better than wget in some cases. Let’s dive in!
Calendars seem simple at first, but no system survives first context with reality. Let’s go through what types of calendars exist and how to support them all in one codebase. Our eventual focus will be on producing a calendar for games — but those more on the Dwarf Fortress end of the spectrum.
We’ve been talking about procedural content generation for a MUD, but it would be rather sad to have a procedurally generated static MUD. A place where nothing ever changes.
If we want to talk about change over time, we need a notion of time.
This is the second post in a series on procedural generation for a MUD, which is a text-only MMO, or a multiplayer Zork. The eventual goal is to create a large, varied MUD containing multiple cities, towns, and wilderness biomes, and a wide array of NPCs and items, as the basis for more personalized and customized work.
Last time, we spoke about placing towers for the city walls. Let’s create the walls themselves.
This starts a series on procedural generation for a MUD, which is a text-only MMO, or a multiplayer Zork. The eventual goal is to create a large, varied MUD containing multiple cities, towns, and wilderness biomes, and a wide array of NPCs and items, as the basis for more personalized and customized work.
Today we’re looking at generating the outline of a city: the city walls.
My reaction to this is a giant NO.
Extracting a browser kernel that runs services on top, that’s great. The rest, not so much. Right now, I have to trust Apple or the KDE devs or Google or Opera or Microsoft that my browser isn’t spying on me. Then I use RequestPolicy and Privacy Badger and sometimes NoScript to ensure the page itself isn’t doing bad things. But with Atlantis, I have to trust that every single website that contains anything I care about has done research on every component it depends on. And I already distrust these websites, by and large.
The examples didn’t have any security on them. They referenced compilers and stuff via unencrypted links. No checksums. No signing. So my bank does due diligence and determines that the parser and renderer and compiler it’s using are secure — but I get MITM’d and all my bank details are siphoned off to Russian hackers. Or the bank uses HTTPS and the server hosting the parser and renderer gets hacked and I get sad.
So instead my bank hosts everything itself. And there’s a bug in the RPC handler that allows arbitrary code execution. I know about it, upstream knows about it, there’s a fix released…but my bank is still using a three year old version and the Russian hackers are starting to feel sorry for me.
Fortunately, the security story is moderately straightforward. We have a central repository of trusted services. You can request specific versions of a service, but the browser doesn’t guarantee that it will honor your version request. For instance, if you request mshtml11.3.7, the browser might give you mshtml11.3.12. In point of fact, we’ll only support major.minor version requests; you always get the most recent patch version.
A service build is automatically retired after eighteen months to mitigate security risks. This is why the browser won’t always honor your version requests. You might have asked for mshtml4.0, but nobody’s been maintaining that for a decade and more, so the browser will give you the nearest equivalent.
Since we’re using a silo for trusted services, we can use a bunch of things like signed builds and certificate pinning to reduce your ability to muck about with my trusted resources.
Finally, Atlantis internally has an RPC mechanism defined. You can post arbitrary data to arbitrary pages. That’s a problem. You need a way to lock that down. Without a means of restricting it, I can construct a page that will fuzz your other open tabs. Perhaps you require a handle to a page in order to send RPCs, and the only way of getting a page ID is by opening it (or receiving the ID via an RPC). Perhaps there are named RPC channels that a page must enroll in, and the browser automatically drops RPCs that aren’t supported.
One big thing that web devs tend to want is analytics. I’m not so keen on being tracked. It’s straightforward in Firefox to reduce tracking: suppress the HTTP referer header, add a pre-request hook that will disallow cross-origin requests contrary to a defined policy, and delete cookies on schedule. Maybe pass the Do Not Track header too.
How do I do that in Atlantis?
I have to trust Atlantis to pass the referer header, then I have to use a local proxy for everything it does. That works okay for HTTP, but it doesn’t work with HTTPS. With HTTPS, the referer header is encrypted and my proxy can’t see it. Or my proxy needs to have a certificate that the browser implicitly trusts for every site, and I have to disable certificate pinning in the browser.
This goes into the general realm of promoting user agency vs promoting developer agency.
Atlantis uses abstract syntax trees as the basis of everything.
Abstract syntax trees let you abstract over different programming languages a little. Not much. You’re stuck with one set of semantics. It’s like trying to implement C on the JVM — you just can’t do it. You can do some of it, but you don’t get pointers and you don’t get structs, so you’ll end up writing a virtual machine on top of the JVM to support those.
So that’s a constraint that limits you a lot. The obvious alternative is LLVM bytecode. Google Chrome’s NaCL, if I recall correctly, uses LLVM, so it’s possible to sandbox it.
The other problem I have with the version presented is the rendering system. It builds a bitmap image and sends it off to the browser. I’m not sure how that will work for accessibility. I’m not sure how to highlight text and copy it that way. It’s good enough for research, not good enough for real life. And if you punt this to the service developer, 95% of them will ignore accessibility entirely, and 30% of them will forget about copy/paste.
If you split up a browser into multiple services running on a base, things get nicer on the technical side.
The core of the browser can just be a few process-oriented APIs, a series of drawing primitives, IO, etc. That’s simple enough to implement on its own.
Independent parties can develop services to implement, say, SVG or MathJAX. And with agreed-upon APIs, I can use the same service on IE and Firefox. This is good for web standards — they can be implemented more quickly, and it’s easier to track down the source of incompatibilities when you can insert the W3C version of MathJAX into Firefox, observe how it renders, and then swap out Trident for Gecko to see if there’s a bad interaction between Gecko and W3C:MathJAX that’s messing up the output.
Then I can implement browser addons as services that the user brings in. For special purposes, when the user allows, pages can do nonstandard things too, implementing their own services. For instance, the Elm programming language provides a moderately different layout system that tends to be pixel-based. (The relatively recent html package offers access to DOM, but the older stuff doesn’t.) That could be implemented as a new rendering service. Or if we find a way to provide sandboxed GPU access, we could get a Unity3D service. Or with DRM, a page could supply a service that converts encrypted audio to WAV.
There’s a lot of possibility here. And I’m sure that James Mickens has considered some of it. A one-hour talk isn’t the best for conveying the full depth of your vision. I’m excited to see his continuing work.
Fresh off the press, my new hit single, Not Pope Innocent!
I think I did it again
I made you believe it’s God’s holy war
I might write papal bulls
But that doesn’t mean I’m your next saint
Cause to win at politics
That is just how I want to beat
Oops, I did it again
Invaded the Jews, got lost in crusades
(Oh one church, true church)
Oops, you think I’ve God’s love
That I was sent from abo-o-ove
I’m not Pope Innocent
You see my pawns are arranged
You’re wasting away
Wishing the cardinals would all vote your way
I snort, awaiting the day
Can’t you see you’re a fool flubbing political plays?
But to win at politics
That is just how I want to beat
(di Sezze, Sezze)
(Pope Alexander! Wait, before you go, there’s something I want you to have.)
(It’s slimy! But wait, isn’t this…?)
(Yeah, yes it is.)
(But I thought the Spear of Longinus was at the bottom of the Dead Sea.)
(Well, Your Holiness, I went down and brought it back for you.)
(I grant you lands and my blessing for your service to the Holy Church.)
Oops I did it again to your church
Got lost in crusades oh synod
Oops you think that I’m sent from above
I’m not Pope Innocent.
(In the interests of historical accuracy, I must point out that Pope Alexander III did not, in fact, call any crusades.)
I’ve been seeing a lot of recommendations for the Firebird’s Son trilogy by Darth Marrs. They praise the worldbuilding. I really don’t see it, and I’m tired of repeating why, so I’m cataloguing it here.
The Firebird’s Son trilogy is a work of Harry Potter fan fiction. The premise is that, in the UK, there is a strongly matriarchal society, there are about three women for every man, and birth rates are rather low. Furthermore, men and women bond the first time a man has sex, and that bond transfers magic power from the man to the woman.
The sheer number of things that don’t make sense in the story is mind-boggling.
Men are rare — or not
Men are rare and therefore valuable, right? Except almost nothing in the story reflects that.
If men were rare, they would be protected, from infancy to death. What we actually see is a surprising degree of callousness. Boys are allowed to play Quidditch, for instance, despite the risk of injury. And men are even allowed to join the DMLE.
Why? It doesn’t make any sense!
Furthermore, malnutrition can erode your magic power, which (as we’ll discuss soon) impacts your ability to form bonds. But nobody checked on Harry to see that he was being well fed.
Men lose magic power
Men are expected to enter a bond and lose most of their magic. Then, if they have enough magic left, they’re expected to enter another bond. And again, and again, until they’re barely able to hold a wand.
(Squibs, meanwhile, supposedly tend to leave magical society because it’s too tough for them to be able to see magic without being able to perform it. How much worse must it be to be able to do so little when only a few years prior you could do so much more!)
Boys need some amount of magic training, otherwise they’re a danger to themselves and others. However, they don’t need as much as women. It’s a waste of their time and their hope to teach them more magic than necessary for them to control their power.
It would be sensible for men to take administrative positions that benefit less from magic. Accounting should be a popular course for boys.
Bonding is spoken of as if it’s serious business. On the ground, people treat it as if it’s important and weighty. At the administrative level, nobody seems to care much about it.
Bonding is the core of family structure, so it’s not something you want to jump into. Also, bonding someone too young will kill them or render them catatonic. So a sensible person would try to keep boys and girls separate, right? At least in unsupervised conditions — kids aren’t going to molest each other under a teacher’s watchful eye.
But Darth Marrs thinks it’s sensible to keep everyone in the same boarding school, on the same Quidditch teams, with easy access to each other. The only vague sop to separating them he institutes is that they can’t have classes together until fourth year — as if kids tend only to be sexually active before they turn fourteen while in a classroom.
The covens (old, powerful groups of witches) tend to control bonding. They would want to crack down on unauthorized bonding, so segregated classes would be high on their agenda. Women who bond without their consent tend to be in for a rough time (though they can gain status from having a powerful husband), and we don’t really see love matches. Plus if there’s a tacit attempt to get men to rebel more often, bonding won’t help.
Really, no authority figure benefits from unauthorized bonding, and all of them should be in favor of gender segregation.
One problem the series tries to drive home is the declining population in Magical Britain. Everyone talks about it, so you’d expect someone would do something about it.
Combine that with the high gender imbalance and the limits on polygyny, and you should naturally get a high rate of halfbloods with magical mothers. Do we? It doesn’t seem like it. If there were, we’d see population growth, since some 70% of the population is capable of bearing young. Instead of needing two children to break even, as we see in real life, two children would give a 40% increase.
If that doesn’t work, men can’t form a second bond without their first bond-holder’s consent. So you can have a high degree of polygyny without lots of bonding, right? Except nobody has thought of that.
If that also doesn’t work, artificial insemination.
And if that also doesn’t work, research other methods of conception. Find a way to terminate bonds without hurting everyone to ensure that men always have bondmates that can bear children. Try something else. Try something!
But nobody tries anything. Nobody cares.
Similarly, Muggleborns tend to die because they don’t have a witch’s breastmilk. But there’s no attempt to locate them and find them wetnurses, which would add to the magical population.
Everything above was major, story-changing stuff, things that would require the plot to be scrapped or at least major portions of the story to be rewritten. There are also plenty of minor things that could be fixed with smaller modifications:
- Eleven-year-olds engage in sexual banter.
- Harry is an Aether, which is rare and unusual and indicates upheaval. Is it worth keeping him around and in possession of his faculties? Why isn’t he tracked more closely?
- There are not just traditions but important social structures around polygyny. The current situation has been around for nine generations. Is that really long enough to create these social structures?
- Kingsley Shacklebolt is a member of the DMLE. Safety aside, does he have enough magic power to do his job?
- Students take 14 hours per week of Muggle Studies. It’s unclear how many years she teaches, but if it’s even two full years, she’s working very long hours. (Recall that there are at least two classes.)
- Lucius Malfoy went to Azkaban. Does it have any impact on fertility? The bonds? If so, it probably wouldn’t be allowed, but it doesn’t seem likely that Dementors would leave you in a good condition for begetting more children.
- Magical people are “genetically incompatible” with Muggles starting when they turn forty. THAT’S NOT HOW GENETICS WORKS.
What can we learn from this?
Overall, it seems like Darth Marrs had an idea of what style of world he wanted and tried to patch it to make it make sense. But he stopped after one set of patches, ending up with something that only bears up under five minutes of scrutiny, if that. This might be better than average, but it’s still lazy or inept, and I can’t tolerate it.
If you want bulletproof worldbuilding, you need to be adversarial about it. You need to be able to ask in one moment “how do I make this world into one that I want?” and in the next “how might people living in this world break it?” You have to turn the pillars of your universe into problems to be solved and then try to solve them.
Or if that’s too much, you can simply say: there are people working on these problems, but they’ve encountered difficulties and have no working results yet.
Darth Marrs didn’t, and the story suffers for it.