Simple Level Streaming

Well, as promised, here's a post (rather lengthy) about the simple world streaming system which I implemented for my game.

But before I begin, I just wanted let you know that this post might get a little technical at times.  So for those of you who are not too interested in the technical stuff, you can either skip over those parts, the post entirely, or ask me questions in the comments section below.  As for those who might want more technical information, well, you can do the same.  :)

A little info about streaming

First off, let me explain a little about world or level streaming.  Essentially, when a game starts up and loads the level you're about to play, it usually loads everything it needs from that level into memory: textures, meshes, audio, animations, object data, etc.   So, you can imagine that if you had a really huge level the memory would fill up quite fast and you'd most likely run out of memory and then, well, weird things would start to happen and the game could even crash.  Memory is not the only thing that could be impacted by huge levels.  Performance can also become an issue, whether it be rendering performance or CPU performance.  Now, there are methods which exist to help increase rendering performance such as occlusion culling, but the more information the occlusion system has to process, the greater the performance hit on your CPU, so it's a trade off.

At this point I could deviate and start talking about other methods to help with CPU and rendering performance, but that's not the focus of this post. 

Now, to get back to streaming.  One way you address the potential issues mentioned above is by using level or world streaming.  Ultimately, the concept behind streaming is simple;  stream (load) in what you need, when you need it, and get rid of it (unload) when it's no longer need.  In the end this will help out with memory, CPU and rendering costs.

My streaming system

As you know (if you've been following my posts),  my game will be mostly about exploration and since this aspect is important, I would like to have fairly big worlds to explore.  Keep in mind that I could have created different sections, and as you move through the various sections of each world, I could have popped up a load screen before loading to the next section.  This works fine and it has been done in many games, even today.  However, I feel that this pulls you out of the experience, and depending on the load times, it could even become frustrating to the player while moving from section to section.

Raise curtain, in comes world streaming...

Although I knew I needed some kind of streaming system, I did not want to make something overly complex or long to make either.  I have much more important things to work on, like the actual game for example. :)  So, I got to thinking and developed something simple that would suit my needs.

World Editor

I created a tool in unity, which I called the World Editor (I know not very imaginative).  This tool is used to create the data for the world streaming system which is then used by the WorldSectorManager in game.

The world editor will automatically detect all objects tagged as WorldSectorObject and generates sectors based off of the information specified in the editor window (covered below).  On top of automatically generated sectors, you can create manual sectors which are not deleted or re-sized by the World Editor whenever you bake them.  

Once the information has been baked, the World Editor will generate the connections for each sector (if enabled) so that the WorldSectorManager will be able to know which sector to load/enable and unload/disable later on.

After the the sectors have been baked and the sector connections have been made, if streaming is enabled, you can use the World Editor to save all your sectors to individual scenes which will later be used by the in-game streaming system (WorldSectorManager).  A world root scene is also generated in order to launch the game from there, leaving your main scene which you use to design, intact.

World Editor window

Here's a brief overview of each item found in the editor and what it does:

• World Center
• This is the center of the worlds bounding volume.
• World Size
• This is the size of the worlds bounding volume (includes all world objects and sectors)
• Use Streaming
• When this is selected it allows you to stream (load/unload) all sectors and their objects from the disk when needed.  If this option is not selected, then the world sector manager simply enables / disables sectors and their objects.  You get some performance gain from this but no memory gain.  This was mostly developed for debugging purposes but can be used otherwise.
• Sector Size
• This is the size (XZ) of automatically generated sectors
• Recalculate World Bounds
• When this is checked off, the world bounds are recalculated automatically every time you bake or save sectors or the world root.
• Auto Generate Sectors
• If this is selected the world editor will automatically generate the sectors it needs based off of the tagged objects
• Build Sector Connections
• If this option is selected then when baking, the connections between sectors will automatically be generated.
• Diagonal Connections
• If this is selected, while building automatic sector connections, diagonal connections will be allowed.
• Bake Sectors
• This generates the world sectors depending on the tagged objects and the world bounds
• Clear ALL Sectors
• This removes all sectors (manual or automatic) from the world root.
• Display Options
• Display World Bounds 
• If selected, while in the unity editor, a box  will be displayed showing the world bounds
• Display World Sectors
• If selected, while in the unity editor, boxes will be displayed showing each sector.
• Display Sector Numbers
• If selected, while in the unity editor, the sector numbers will be displayed.
• Bake Before Saving
• If this is selected, when saving either sectors or the world root, the editor will bake the sectors.
• Save all to scenes
• When streaming is enabled, this will save the world root and all sectors and their contained objects to individual scenes.
• Save world root scene
• When streaming is enabled, will only save the world root to a scene

WorldSectorManager

This manager is what takes care of streaming the world, its sectors and objects.  When the game launches, the manager will try to find a world root within the scene.  If one is found, then the WorldSectorManager starts doing its job.  You simply tell the manager where you are in the world and it will take care of streaming asynchronously or enabling / disabling sectors, depending on what you selected in the world editor.

World Root

This  component is added automatically to the world root by the World Editor.  It represents the worlds' root node.  It contains all the functionality related to the world root as well as the sectors, all the fields and configurations which are used by the World Editor.

Here's a brief overview of each item found in the editor and what it does:

• Open World Editor
• Opens the world editor
• Add Manual Sector
• This allows you to add a manual sector to the world.
• Manual Sectors
• You can remove the manual sectors by clicking the red X next to manual sector in the list (none in image above).
• Scene name
• Displays the streaming scenes' name
• Path
• Displays the path where the streamed scene is saved
• Save to root scene
• Saves the world root to its streamed scene

World Sector

This component is added automatically to each sector by the World Editor.  It represents a sector found within the world.  It contains all the functionality related to a world sector as well as all its connected sectors and contained world objects.

Here's a brief overview of each item found in the editor and what it does:

• Convert To Manual Sector
• This button only appears on automatically generated sectors.  If you want to convert this sector to a manual sector so it will no longer be touched by the automatic process, simply press this button.
• IsGlobal
• A sector can be flagged as being global.  This means it will never be unloaded or disabled.
• Size
• The size of the sector bounds
• Center
• The center of the sector bounds
• Diagonal Connections
• If this is selected, the sector will allow diagonal connections to it when baking in the World Editor.
• Build Connections
• You can rebuild only the connections for that sector using this.
• Scene name
• Displays the streaming scenes' name
• Path
• Displays the path where the streamed scene is saved
• Save to sector scene
• Saves the sector to its streamed scene

Playing in the Unity Editor

As mentioned previously, when using streaming, the world root is actually saved to another scene.  This is done because if we were to load the original scene, it would actually load everything, then unload what it doesn't need.  That's extremely inefficient and defeats the purpose of streaming.

So, while in the editor, if the current scene has a world root, it determines if you already are in the world roots' streamed scene.  If so, nothing needs to be done and the game simply launches.  However, if you are not in the world roots' streaming scene, it will open the streaming world root scene and run the game from there.  When you stop playing, it will bring you back to the scene you were currently editing.

When you want to load levels through the regular flow of  your game, then it is important to load that levels' root scene directly instead of the original scene with everything in it.

Final Thoughts

For those of you who are wondering why I did this when Unity 5 has announced that they will have level streaming, well, my answer to you is this:

I have worked with Unity for many years now and I know that it takes them forever to release versions.  On top of that, when they first come out with new features, they are sometimes incomplete or very buggy.

That being said, for the moment, it is not in my plans to switch over to Unity 5 upon its release.  :)

I hope that the information I shared was to your liking.  If you have and questions or comments, don't hesitate.

Did someone say art?

This week I worked on a simple world streaming system in Unity.  Unity does not have this kind of functionality built in, fortunately it exposes certain tools which makes it possible.  So, as of yesterday, I am now able to have streaming worlds within Unity!  Yay!  I'll try to post the details about this later.

But enough about that for now.  This post is actually more about art.  As I mentioned in an earlier post, the art style which I chose for my game is Low Poly art.  There is a great tool available on the unity asset store which helps you convert unity terrains into low poly terrains, and convert your objects into low poly objects.  Obviously, I purchased the tool.  :)

If you want more information about this great tool, called PolyWorld-Woodland,  you can find it here:

http://qt-ent.com/PolyWorld-Woodland/


So, with the help of the aforementioned tool and, my brother who has been creating some assets for the first world, here are a few screen shots of some of these assets.  Keep in mind these are still work in progress.

A nice little well.

Flower bush showing varying levels of "well being".  There are
many levels which are controlled at run-time.

One of the trees showing varying levels of "well being".  There
are many levels which are controlled at run-time.

So here it is.  Hope you like what you see.  Let me know your thoughts.

Still alive

I haven't posted anything lately.  I've been pretty much focused on gameplay mechanics.  I've started working on my prototype as of last week.  It's one week earlier than planned, but I felt like doing that.

I'll try posting more and more updates (real ones) as the project moves forward.   I'm still getting used to this posting stuff.  :)

Game Production

Maybe some of you are wondering or thinking that my previous posts, and most likely this one, are kind of random.  Maybe to someone on the outside it may seem that way, and in a way it is, but, ultimately there is a method to my madness (or so I believe :)).  Before I explain, let me talk a little about my past.

So here's the thing.  I've been working as a programmer in some form or another (programmer, senior programmer, lead programmer, etc.) for close to twenty years now.  Around ten of those years have been in the game industry, which are the most pertinent to this post.

During my time in the industry I've worked on many games of different sizes, on different platforms (handhelds, consoles, PC) and different team sizes.  All these games had productions, and in my mind, all these productions had problems.  One of them was they operated in a way that did not promote creativity but suppressed it.  Now keep in mind, this is only my opinion and you may talk to people who worked on those same games and productions, which hold a different view.

Here is how I see it.  Making games is a creative process.  It’s not an assembly line in which things are done one after another, until it reaches the end with the finished product, never breaking the chain.  This process does not cater to creativity.  Most game companies say: “We use the agile process, we rock!” (Okay, they don’t exactly say that, but you get my point J).  This couldn't be farther from the truth.  To them being Agile is being able to replace a 3 day feature you've been working on for the last 2 days with another 3 day feature, which appeared out of nowhere and, getting it done in the 1 day you have left.  Everything is scheduled, one after the other, and we (people on production), sit there, like good little workers on the assembly line and hack away at the tasks given to us, until it changes or it’s done, regardless if they inspire us or not.  Don’t get me wrong, we have all done boring tasks, and will have to again during our careers. At its core, this process is probably even worse than the assembly line and is ultimately some hybrid between the waterfall and the agile approach.  It does not promote creativity whatsoever, but stress and useless pressure which could have been avoided (but that’s another topic altogether J)

So here is where I come back to my posts, and my game production.  I’m making a game, my own game.  I’m doing this because I love games, and because I've always wanted to make my own.  I’m doing this because to me, making a game should be as fun as playing one, or at least I believe it should be.  Think about it.  How can your game be fun, inspired, if you yourself did not have fun or get inspired while making it?

I know what my game will be, its design is complete (for the most part), I have a plan, tasks and tentative milestones.  I have assigned priorities to features and tasks, and I can track the progress I’m making on a day-to-day and milestone-to-milestone basis.  With that said, if I feel like working on one thing one day, then something else that inspires me more on another day, then I do it, all within reason of course.  Ultimately, all of it will get done in the end, and will be much better if I was inspired and had fun while doing it.

Yes this may seem utopic and one might say, “you’re alone, of course it could work”, but I believe if you do things right, plan, have fun and be inspired, it can be done no matter how many people you are.

So on that note, stay tuned for my next, potentially “random” post!

Procedural vfx test

Here's a little test I did today to make a procedural tentacle like effect which I will be needing for my game.   Keep in mind, this is mostly to show the "tentacle" feel and it does not reflect a final look (textures, color, particles, etc.).  

For the moment I can play around with many settings in order to change its look and feel.  Let me know what you think.

I apologize for the quality of the video.  Will try to make some better ones next time.