Simple Level Streaming

Well, as promised, here's a post (rather lengthy) about the simple world streaming system which I implemented for my game.

But before I begin, I just wanted let you know that this post might get a little technical at times.  So for those of you who are not too interested in the technical stuff, you can either skip over those parts, the post entirely, or ask me questions in the comments section below.  As for those who might want more technical information, well, you can do the same.  :)

A little info about streaming

First off, let me explain a little about world or level streaming.  Essentially, when a game starts up and loads the level you're about to play, it usually loads everything it needs from that level into memory: textures, meshes, audio, animations, object data, etc.   So, you can imagine that if you had a really huge level the memory would fill up quite fast and you'd most likely run out of memory and then, well, weird things would start to happen and the game could even crash.  Memory is not the only thing that could be impacted by huge levels.  Performance can also become an issue, whether it be rendering performance or CPU performance.  Now, there are methods which exist to help increase rendering performance such as occlusion culling, but the more information the occlusion system has to process, the greater the performance hit on your CPU, so it's a trade off.

At this point I could deviate and start talking about other methods to help with CPU and rendering performance, but that's not the focus of this post. 

Now, to get back to streaming.  One way you address the potential issues mentioned above is by using level or world streaming.  Ultimately, the concept behind streaming is simple;  stream (load) in what you need, when you need it, and get rid of it (unload) when it's no longer need.  In the end this will help out with memory, CPU and rendering costs.

My streaming system

As you know (if you've been following my posts),  my game will be mostly about exploration and since this aspect is important, I would like to have fairly big worlds to explore.  Keep in mind that I could have created different sections, and as you move through the various sections of each world, I could have popped up a load screen before loading to the next section.  This works fine and it has been done in many games, even today.  However, I feel that this pulls you out of the experience, and depending on the load times, it could even become frustrating to the player while moving from section to section.

Raise curtain, in comes world streaming...

Although I knew I needed some kind of streaming system, I did not want to make something overly complex or long to make either.  I have much more important things to work on, like the actual game for example. :)  So, I got to thinking and developed something simple that would suit my needs.

World Editor

I created a tool in unity, which I called the World Editor (I know not very imaginative).  This tool is used to create the data for the world streaming system which is then used by the WorldSectorManager in game.

The world editor will automatically detect all objects tagged as WorldSectorObject and generates sectors based off of the information specified in the editor window (covered below).  On top of automatically generated sectors, you can create manual sectors which are not deleted or re-sized by the World Editor whenever you bake them.  

Once the information has been baked, the World Editor will generate the connections for each sector (if enabled) so that the WorldSectorManager will be able to know which sector to load/enable and unload/disable later on.

After the the sectors have been baked and the sector connections have been made, if streaming is enabled, you can use the World Editor to save all your sectors to individual scenes which will later be used by the in-game streaming system (WorldSectorManager).  A world root scene is also generated in order to launch the game from there, leaving your main scene which you use to design, intact.

World Editor window

Here's a brief overview of each item found in the editor and what it does:

• World Center
• This is the center of the worlds bounding volume.
• World Size
• This is the size of the worlds bounding volume (includes all world objects and sectors)
• Use Streaming
• When this is selected it allows you to stream (load/unload) all sectors and their objects from the disk when needed.  If this option is not selected, then the world sector manager simply enables / disables sectors and their objects.  You get some performance gain from this but no memory gain.  This was mostly developed for debugging purposes but can be used otherwise.
• Sector Size
• This is the size (XZ) of automatically generated sectors
• Recalculate World Bounds
• When this is checked off, the world bounds are recalculated automatically every time you bake or save sectors or the world root.
• Auto Generate Sectors
• If this is selected the world editor will automatically generate the sectors it needs based off of the tagged objects
• Build Sector Connections
• If this option is selected then when baking, the connections between sectors will automatically be generated.
• Diagonal Connections
• If this is selected, while building automatic sector connections, diagonal connections will be allowed.
• Bake Sectors
• This generates the world sectors depending on the tagged objects and the world bounds
• Clear ALL Sectors
• This removes all sectors (manual or automatic) from the world root.
• Display Options
• Display World Bounds 
• If selected, while in the unity editor, a box  will be displayed showing the world bounds
• Display World Sectors
• If selected, while in the unity editor, boxes will be displayed showing each sector.
• Display Sector Numbers
• If selected, while in the unity editor, the sector numbers will be displayed.
• Bake Before Saving
• If this is selected, when saving either sectors or the world root, the editor will bake the sectors.
• Save all to scenes
• When streaming is enabled, this will save the world root and all sectors and their contained objects to individual scenes.
• Save world root scene
• When streaming is enabled, will only save the world root to a scene

WorldSectorManager

This manager is what takes care of streaming the world, its sectors and objects.  When the game launches, the manager will try to find a world root within the scene.  If one is found, then the WorldSectorManager starts doing its job.  You simply tell the manager where you are in the world and it will take care of streaming asynchronously or enabling / disabling sectors, depending on what you selected in the world editor.

World Root

This  component is added automatically to the world root by the World Editor.  It represents the worlds' root node.  It contains all the functionality related to the world root as well as the sectors, all the fields and configurations which are used by the World Editor.

Here's a brief overview of each item found in the editor and what it does:

• Open World Editor
• Opens the world editor
• Add Manual Sector
• This allows you to add a manual sector to the world.
• Manual Sectors
• You can remove the manual sectors by clicking the red X next to manual sector in the list (none in image above).
• Scene name
• Displays the streaming scenes' name
• Path
• Displays the path where the streamed scene is saved
• Save to root scene
• Saves the world root to its streamed scene

World Sector

This component is added automatically to each sector by the World Editor.  It represents a sector found within the world.  It contains all the functionality related to a world sector as well as all its connected sectors and contained world objects.

Here's a brief overview of each item found in the editor and what it does:

• Convert To Manual Sector
• This button only appears on automatically generated sectors.  If you want to convert this sector to a manual sector so it will no longer be touched by the automatic process, simply press this button.
• IsGlobal
• A sector can be flagged as being global.  This means it will never be unloaded or disabled.
• Size
• The size of the sector bounds
• Center
• The center of the sector bounds
• Diagonal Connections
• If this is selected, the sector will allow diagonal connections to it when baking in the World Editor.
• Build Connections
• You can rebuild only the connections for that sector using this.
• Scene name
• Displays the streaming scenes' name
• Path
• Displays the path where the streamed scene is saved
• Save to sector scene
• Saves the sector to its streamed scene

Playing in the Unity Editor

As mentioned previously, when using streaming, the world root is actually saved to another scene.  This is done because if we were to load the original scene, it would actually load everything, then unload what it doesn't need.  That's extremely inefficient and defeats the purpose of streaming.

So, while in the editor, if the current scene has a world root, it determines if you already are in the world roots' streamed scene.  If so, nothing needs to be done and the game simply launches.  However, if you are not in the world roots' streaming scene, it will open the streaming world root scene and run the game from there.  When you stop playing, it will bring you back to the scene you were currently editing.

When you want to load levels through the regular flow of  your game, then it is important to load that levels' root scene directly instead of the original scene with everything in it.

Final Thoughts

For those of you who are wondering why I did this when Unity 5 has announced that they will have level streaming, well, my answer to you is this:

I have worked with Unity for many years now and I know that it takes them forever to release versions.  On top of that, when they first come out with new features, they are sometimes incomplete or very buggy.

That being said, for the moment, it is not in my plans to switch over to Unity 5 upon its release.  :)

I hope that the information I shared was to your liking.  If you have and questions or comments, don't hesitate.