Multi tool use

Recommended process for saving Custom C# Classes (containing lists/arrays of various size) other than binaryformatter.serialize?

I would like users to be able to create grid-style 'maps' in my application, save them to disk and then share them with others. Among basic map data I would like the user to enter their name so as to 'claim' their work.

For this to be a little bit more secure I was thinking of saving the map class (with the artists name, map dimensions, other useful options and the flat/2D array/list [delete as necessary]) into binary. So, XML's (being human readable) are probably not what I'm going to be using... but I'm not completely dead set against them yet, just cautious to stop 'stealing'.

I've been using the BinaryFormatter.Serialize method, however as the size of my tiledata grows from 200*200 tiles to 300*300 tiles the time it takes to deserialize grows from 6.5 seconds to 33 seconds, and continues to increase in an exponential fashion.

I would, ideally, like to be able to store mapdata of 2000*2000 if possible.

So, apart from learning compression techniques (which I will be doing anyway to reduce file sizes) I was wondering if there were any other serializers that are recommended that I use. I've seen that there are some 3rd party ones available, but need to figure out if they're compatible with Unity (yes... I'm a self taught noob gamedev).

Any assistance would be greatly appreciated.

Current test code added for clarity:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using System;
using System.IO;
using System.Runtime.Serialization.Formatters.Binary;

public class FileReaderTest : MonoBehaviour
{
public int xlimit = 300;
public int ylimit = 300;

// Use this for initialization
void Start()
{
Save();
}


// Update is called once per frame
void Update()
{
if (Input.anyKeyDown)
Load();
}

private void Save()
{
BinaryFormatter formatter = new BinaryFormatter();

FileStream file = File.Open(Application.dataPath + "/saved.data", FileMode.OpenOrCreate);

List<TestDataBaseEntry> mapTileData = new List<TestDataBaseEntry>();
List<TestDataBaseEntry> extraMapTileData = new List<TestDataBaseEntry>();

for (int x = 0; x < xlimit; x++)
{
for (int y = 0; y < ylimit; y++)
{
// Simulating random data
mapTileData.Add(new TestDataBaseEntry(TileState.Filled, x, y));
}
}

for (int x = 0; x < 5; x++)
{
for (int y = 0; y < 3; y++)
{
extraMapTileData.Add(new TestDataBaseEntry(TileState.Ignore, x, y));
}
}


TestMapFile newFile = new TestMapFile("Mike", xlimit, ylimit, mapTileData, extraMapTileData);

formatter.Serialize(file, newFile);
file.Close();

Debug.Log("saved");
}

private void Load()
{
float starttime = Time.realtimeSinceStartup;
if (File.Exists(Application.dataPath + "/saved.data"))
{
using (FileStream file = File.Open(Application.dataPath + "/saved.data", FileMode.Open))
{
BinaryFormatter formatter = new BinaryFormatter();

TestMapFile retrievedTestMapFile = (TestMapFile) formatter.Deserialize(file);
}
}

Debug.Log("Loaded");
Debug.Log("The process took " + (Time.realtimeSinceStartup - starttime));
}

}

[Serializable]
public class TestDataBaseEntry
{
public TileState tileState;
public int x;
public int y;

public TestDataBaseEntry(TileState newTileState, int newX, int newY)
{
tileState = newTileState;
x = newX;
y = newY;
}
}

[Serializable]
public class TestMapFile
{
public int xSize;
public int ySize;
public List<TestDataBaseEntry> mapTileData = new List<TestDataBaseEntry>();
public List<TestDataBaseEntry> mapExtraTileData = new List<TestDataBaseEntry>();
public string createdBy;

public TestMapFile(string artist, int newXSize, int newYSize, List<TestDataBaseEntry> newDB, List<TestDataBaseEntry> newExtraDB)
{
createdBy = artist;
xSize = newXSize;
ySize = newYSize;
mapTileData = newDB;
mapExtraTileData = newExtraDB;
}
}


2 Answers
2

Look at protobuf-net. It is not so readable, quite fast and takes less disk space.
https://github.com/mgravell/protobuf-net

Okay, I've taken advice on board from a previous post I had on the Unity forums, as well as considered some of the points that were said here, and created a few test scripts - here's my findings.

First of all, following on from something @Scott Hannen mentioned in a comment (that was also mentioned in the previous post on forum.Unity.com) I tried make my own de/serializer, I stripped the information to be put into (and read back from) the file and got the time down to 0.13 seconds for enough information to recreate the terrain for a map of 1920*1080 size. The data I was working with was just a string, 2 ints and a flat array of bools.

So I then went back to look at the BinaryFormatter again. Straight away the biggest difference I had to address so as to bring the comparisons inline with each other was to change the list of the custom class TestDataBaseEntry into an array of bools (which is what I used with the binarywriter). This alone brought the time of reading the file down from 33.3 seconds for a map size of 300*300 to 11.1 for a map size of 1920*1080!!! A massive imrpovement - I'm guessing that some of the BinaryFormatter's process is a lot of background checks and error checking?

So it's still not as quick as my stripped back custom serializer. However it's still not a true comparison either since I'm still de/serializing an actual class with BinaryFormatter, whereas I'm de/serializing just the components of that class individually using BinaryWriter/BinaryReader.

So I still have some more testing to do, but the solution for me, so far, seems to be to uncomplicate the information I'm trying to write to file and remove as many custom types(?) as possible. It's quite possible that I'm reading the wrong answers from this... but I'll crack on and see what I can do.

Many thanks to all who have replied and provide other avenues for me to look into. I will still look into these further so as to expand my knowledge... plus one of them (like the protobuf-net) might still be a better solution to implement ;)

Again, thank you. Laters.

UPDATE - The following morning...

So I ran another test, this time stripping the custom class out altogether and simply serializing and deserializing the relevant part straight to and from the file. Now that BinaryFormatter isn't having to deal with it in a serializable custom class I'm now getting a reading time of 0.15 seconds - which is the same as if I was using my own custom serializer.

So, I have learnt something very valuable with all your help - serialize basic type and not custom classes!

Again, many thanks ;)

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.