Coding Behind the Cube #3 - Build Server

[DeVan0_0]

Nice to see how it works!

 

[D3monRoyal]

Hello again CubeCrafters!

In this edition of Behind the Cube, we're doing a post on the marvelous and magical Build Server!

Wait... A build server?

It can't be that hard - a build server is just creative mode right? It's only building a few maps!​

You may think this, and you wouldn't be wrong!

But also no. So much no.

Yes, you can have a single server in a flat world with only creative mode, but it's not the easiest way, and it's not the most magical way. It's like trying to play Minecraft with the blindness potion effect on.

Some of you might like it, but it's not really that efficient, is it? If I tried to do that to our build team, they would chase me down!

The servers we use to build are split into two, Build Server and Marketplace. They're pretty similar, but Marketplace is designed for the Bedrock Store where we sell our maps, and I think I like our Build Server more because it has all the cool tools that Marketplace just doesn't need.

In this top-secret reveal, we'll be discussing the Build Server.

What does every build server need?

• Map creation tools, such as WorldEdit and VoxelSniper - We love those plugins and know it inside and out!
• World management - We don't want to build all our maps in the same world! We want separate worlds!
• Security - This is a creeper free zone! No trolls allowed!
• Backups - We don't want to lose our work when we trip over wires!
• Aesthetics - We need that colourful chat & ranks, so we know who's responsible around here!
• Fancy Configuration Tools - We want to configure our games easily!
• Map Releasing - We want to distribute completed maps across our server!

Of course, this is our own list of stuff we think is important. Other build servers may add or remove things from this list for what they find important.

We're going to skip over map creation tools such as WorldEdit and VoxelSniper. We want to brag talk about our stuff! Not theirs!

So, that brings us to...

World Management

Every great build server has build teams scurrying over it like ants, massive structures rising in the span of a single day, even as forgotten tombs spread out beneath the sands. But how do we keep track of it all?

You can't simply build everything in a single world; it makes it hard to find places and creates huge world files! It's not sustainable for us as developers, and it's not efficient for the builders!

For this, the obvious answer is to use lots of worlds.
Every gamemode has its own world with its own maps; separating each map into its own world does sound like it'd be a better idea on paper. But then we're introducing a lot more work for no real reason! So yes, each world contains a range of maps for a specific game type.

But even this isn't enough for us, and you are sure to find lots of servers where they use multiple worlds. But think, what happens if you always keep those worlds loaded? Of course, your server is going to suffer in performance!

In our homebrew Build Server, our management system has been built up from scratch, so every world is instantly loaded on demand and, after a minute of inactivity, unloaded again.

It would come as a surprise if you told our build team that the worlds were unloaded before they teleported to that world because the world loads far too fast. This isn't magic on CubeCraft's side, though; you can find similar performance if you tweak settings all over and use good hardware.

But here's another thought. How many worlds do we have anyway?

Oh dear... As of this post, we have 293 worlds. Many of them are temporary worlds, some of them are personal worlds, some are idea worlds, others are actual worlds where we create games and more.

I'm really happy I don't have to keep all of them loaded...

But wait, how do I join the right world? I want to modify some SkyWars maps really quickly!

Oh boy! Just wait until I introduce some more fancy CubeCraft stuff. We have command- and chat-based support and inventory management support!

​

Every builder is given a nether star that, on interaction, will open a menu to configure their settings, pick a world, create a world and more!

The above picture is what I see when I open the menu and hover over a few of the worlds. I'm on page 2, and the items in the row above my mouse lets me filter and sort worlds so that I can find the world I really want to join!

This can be somewhat inefficient though - I know the world name, but I can't find it in the heaps of worlds that we have! So if we're experienced, we just run /world sgmcofinal - And it plonks us right in that world instantly. Pretty neat.

Of course, we can also set some more special settings in this world. The next picture is how we can turn off mobs spawning, change the time, the weather, even another menu to toggle physics individually for a wide range of things! All of this, per world!

I'm not sure why I bothered with some of these settings though, our build team doesn't seem to appreciate that you can place redstone on air, and the redstone will still work just fine. I might've gone too carried away...

View attachment 191633
View attachment 191634

Basic control over a world is a requirement for our build teams, they don't want to fight against zombies when trying to make that next map after all.

Although... That does sound interesting...

Safety and Security

Now, this is a big one; who would want to see their world burn after all?

Each builder must be given a rank on the Build Server that controls what they can do before they're allowed to join, and even then, they still must pass a security check every few hours when they try to join. Two Factor Authentication is important, and if someone's account is compromised, then we don't want to worry about secret projects being leaked or players finding random signs in the lobby with "I was here" written. Not that I would do that. Haha...

This is our /whitelist, something you should enable for any server you want to protect from strangers.

On our server, we try not to limit our builders and instead allow them to unleash their creativity. That's why we don't restrict limitations on WorldEdit. But if you're asking for the power to destroy worlds, we might need to take a second look and see if you're hissing and threatening to explode.

Backups

Unfortunately, many servers don't use backups; this can be catastrophic when something goes wrong. Your server crashes, a fire burns down your house, or you accidentally spawned in half a million TNT. It's ok; we've all done that. Even deleting worlds, though there's a massive warning that says that this world will be deleted permanently... We can sort that out as well!

The most commonly used backup system in Minecraft Servers is normally a script that runs every hour that saves the worlds to a compressed zip file, and these files are normally deleted after some time to prevent them from taking up too much space.

Those files are often huge, and restoring a backup can be complicated as you need to take your server down during the process. But this is a small sacrifice if it means you didn't lose your hours and hours of work forever.

Our own backup system however, is run every 15 minutes. It takes a snapshot of every single world that has been modified since the last backup. Online, or offline.

These snapshots are compressed individually into their own compressed files and saved to an offsite location. If you intend to follow in our footsteps, remember you will need to ensure that the server doesn't try to read or write to the files while creating the world's snapshot! No one likes a corrupted backup.

After we've done this, we unlock the files and inform the builders that yes! We made yet another successful backup!

Just making backups isn't enough, though; sometimes, we want to use those backups.

View attachment 191638

So with a quick command, we look for that backup we want...

View attachment 191639

Then we select it!

A few short seconds later, that backup has been downloaded, and a new world created!

Of course, we could also choose to restore it in the current world, or even download the backup directly for the higher-ranked builders.

Aesthetics

We want a pretty server, and we want it to do things easily.

View attachment 191640

You can see what worlds are loaded in the sidebar, how many players are online, who's sleeping on the job, where the spawn location is, and I'm even holding an item that will open a server management interface if I right-click with it!

View attachment 191641​

Different ranks for different people, chat letting you know if you've loaded or unloaded a world, when you've backed up a world, and much more!

View attachment 191642

Many things go into making the build server a more enjoyable place to stay, with commands to help the builders go about their day to something as simple as disabling kicks when they try to fly too fast.

Releasing a map

After building our maps, we then need to test them and release them.

The process is very simple, just a /release command we can release to either our development/test network or the main network for everyone to enjoy! If we want to test the map on our test network, we click type /release.

View attachment 191643

Then we click on Release on Dev...

View attachment 191644
Then click on the light purple this, and it should automatically release!

Of course, if we want to release it on production, it gives us another message...

View attachment 191645

... With magic characters on both sides of that red text, it's hard to ignore that you're about to release something we're not ready for!

The entire process looks like this:

View attachment 191646

How does it work? Well, I'm glad you asked.

After confirming the release with a code that expires quickly, the world files are locked and compressed into a file.

We then open an SFTP stream which lets us transfer files over an encrypted connection to its destination, all done behind network firewalls where no one can spy on us. Sometimes I even send it over multiple proxies so that we're extra secure. I'm told not to do that, but what does our network security team know...

The file is uploaded to a temporary location on the destination server. Once the upload is complete, we notify the file distribution centre of the file's existence, and it handles it from there! Using some form of magic beyond my imagination to push the map across hundreds of connections so that we can fix that one missing block!

Configuration

Oh boy, I knew I forgot something!

But time is running out, the clock is ticking, and I think I can hear the squeaking of an angry rabbit that doesn't want me to reveal its secrets.

Here's a quick teaser on what it looks like:

View attachment 191647

If there's sufficient interest, we'll make another blog post all about the process of creating our game maps and how the configuration system works!

Thanks for reading!

Such an awesome things!

 

[xbuCKshoTZ]

just want to say i use to play a little on this server in 2017. i actually forgot about it haha. me and my son started playing MC together recently and i must say this server is what i envisioned a MC server to be like. its amazing and love everything about it. keep up and the inside look is awesome to see how this stuff goes together.