|
Elenesski Object Database
EODB and NEODB
The Memento Pattern Database for Unity
|
|
Elenesski Object Database
EODB and NEODB
The Memento Pattern Database for Unity
|
The following Usage Guide exists in order to help you get up to speed with the the Elenesski Object Database.
Everything in the system is focused around a database. Creating a database is easy, you simply say:
EODB myDatabase = new EODB("C:\Documents\Data");
EODB will then prepare the environment and allow now load or save objects to the database. The database keeps an in memory copy of the database, and when you want to persis the data out you say:
myDatabase.CommitToFileSystem();
This command writes the database copy back to the hard drive. If you want to make a new copy of the data while saving, you can say:
myDatabase.CommitToFileSystem("C:\Documents\NewData");
By default, when you open a database with:
EODB myDatabase = new EODB("C:\Documents\Data");
The entire database is loaded into memory. This is also known as a cached database. While this strategy is okay for most cases, large databases can cause issues on mobile platforms, therefore you can open the database non-cached:
EODB myDatabase = new EODB("C:\Documents\Data",null,false);
This creates a duplicate of the "Data.db" file called "Data.tp", and object changes are written in/out of this temporary file.
When you .CommitToFileSystem, the platform reads the .tp file and writes a new Data.db file with all of its indexes. Note, the ".tp" file is not deleted after a save, allowing you to save multiple copies of a database.
The database is stored as a single file. You can index your data to make it easier to search later. Each index requires a separate file. The name of the index is the extension added to the file root. For example, if the path is "C:\Documents\Data" and you create an index called "total", you will find two files in "C:\Documents" called:
Data.db - The database containing all your data. Data.total - Your custom index called "total".
If you open the database, non-cached, see above, there is a third file too, called:
Data.tp - The cached database containing working copies of your data.
Relational database indexes create a dictionary of unique non-null values found in a column of a table. While EODB indexes can do that, the feature is more powerful:
To begin, you need to declare which indexes are available in your database. This is done when you open the database:
EODB myDatabase = new EODB("C:\Documents\Data",new Dictionary<string, Type> {
{"cdname",typeof(string)}
});
This declares a custom indexes called "cdname". What it means is that the index called "cdname" will can contain "string" values. Each string value is now associated to your object, via it's OID, or Object Identifier.
When opening the database, the entire index is loaded into memory, regardless of whether the database is cached or not. While there is no limit as to the number of indexes, indexes can chew up a lot of memory.
Only supported types (see EODBTypeIO.Initialize()) can be used for an Index. Note: MemoryStream is a supported type, however, we recommend against using it as a type of index because it can result in a lot of used up memory.
More on indexes later on.
To read/write object data, what you write must resolve to a known type. It's generally easy to extend the types provided and it's also possible to write type converters, where the internal object format is different than what is in the database (it's converted as it's written to the database, and converted back when it's read from the database):
Out of the box, these types are supported, including Arrays for all of these types
Supported types are declared in the EODBTypeIO.Initialize() method. Each supported type is declared by calling this method from Initialize():
private static void Add<T>(Func<Stream, object> aRead, Action<Stream, object> aWrite) {}
For example, an Integer is declared this way in Initialize():
Add<int>(ReadInt, WriteInt);
And defined this way:
#region --- INTEGER ---
internal static object ReadInt(Stream aStream) {
byte[] BYTES = Read(4, aStream);
return BitConverter.ToInt32(BYTES, 0);
}
internal static void WriteInt(Stream aStream, object aInt) {
Write((int) aInt, BitConverter.GetBytes, aStream);
}
#endregion
Extending the types is relatively easy to do. You simply add the type you want to read/write to the Initialize method and then write the appropriate low-level methods to read/write the type value to/from the stream. For example 3 lines of code to read/write Quaternions:
#region --- QUATERNION ---
public static object ReadQuaternion(Stream aStream) {
return Structure.Read<float, float, float, float, Quaternion>(aStream, (X, Y, Z, W) => new Quaternion(X, Y, Z, W));
}
public static void WriteQuaternion(Stream aStream, object aQuaternion) {
Quaternion QUATERNION = (Quaternion) aQuaternion;
Structure.Write(aStream,QUATERNION.x,QUATERNION.y,QUATERNION.z,QUATERNION.w);
}
#endregion
In some cases it may be necessary to write a type converter which converts aspects of your object into oe format that you persist into the database and then back again. String is often used in this case. For example, you want to create data as a JSON string, then parse it as it comes back to load into the database..
Type conversions occur in a class's Definition() method, and this is described in more detail in the section "Class Definitions ::: Structure". In a nutshell, when you describe your structure, you provide a function that retrieves the data to be stored (YOUR DATA -> database string) and an action that accepts the stored data that you transform into your object (database string -> YOUR DATA).
Each object within EODB has a unique identifier called an OID (Object IDentifier). There are many situations where the data you are storing is for a singleton. A single object representing some aspect of your game, such as, audio settings. In these cases, you need to hard-code the object identifier, and when you do this, this is referred to as Static Data. While you can assign any non-zero OID, by convention, negative value OIDs are considered static data and positive OIDs are considered dynamic.
It's recommended that you create a const for all static data objects:
public const int cAudioSettings = -10; public const int cVideoSettings = -11; public const int cMusicSettings = -12; public const int cGameSettings = -13;
When the data is dynamic, you will need an index to find the objects, since you cannot predict what the OID will be that was assigned to an object.
Associated with each object is a Version. Generally you don't need to worry about Versions, but they exist to handle object variations. Since EODB doesn't declare tables, columns or rows, each stored object is a collection of attributes gleamed from the object and persisted in the database.
One immediate problem arises from this strategy; how do you load objects that were created before you made changes to your game? For example, game version 1 created an object with 3 fields and game version 2 created 4 fields. How do you differentiate the original version of the data with the newer version of the data? By storing an evaluating an explicit version identifier.
The second way you might use versions is to make your definition files easier. Since objects can take many different forms, it might be easier to say this form is version 1, this other form is version 2, and so on.
The version number is part of the IEODBClass interface, and is assigned when you load an object or prior to saving it, such as:
public void DBSave() {
this.Version = 2;
new EODBDescriptor(this);
}
Every object in the database has a header. The header contains 4 attributes:
After these attributes are written out, the order you "Define" your class and the number of bytes you need for the object defines how much information is stored in the database for your object. This means that if you had a string with 25 characters in it, then changed it to 250 characters, then your object becomes 250 characters long (plus 4 bytes for the string length) and the space needed for the header. The database automatically handles where the object is stored, allowing it to grow and shrink over time without wasting space by objects that change size over time.
Depending on your needs, you may need to know an object's version to load it properly. You will most likely use the class name in in combination with a Factory when you have indexes that have OIDs from many different objects. 99% of the time you won't need Version or Class Name most of the time. They exist so that there is a standard way to deal with objects.
You can retrieve the object header using your database:
// returns null if -10 is not a known OID in the database. EODBClassHeader myHeader = myDatabase.GetHeader( -10 ); // where -10 is the OID
Any class in your solution can be used to read/write from a database. All you have to do is implement the IEODBClass interface:
public interface IEODBClass {
EODB Database { get; }
int OID { get; set; }
byte Version { get; set; }
string ClassName();
void Definition(EODBDescriptor aDescriptor);
void DBSave();
}
public EODB Database { get { return GlobalStatic.EODBDatabase; } }
Otherwise, you will need to assign Database in your object's constructor or if you are implementing this with MonoBehaviour with your script's Awake() logic, and have some way to set it by declaring the Database with a setter: public EODB Database { get; set; }
When writing your object to the database, EODB stores binary data without any header information. For example writing two floats (4 bytes each) will result in 8 bytes stored in the database. If you subsequently read those 8 bytes as a long instead of 2 floats, you will get completely erroneous data. To ensure that you are reading the attributes in the same order you wrote them, there is a single method in a class implementing IEODBClass, called Definition(). This method defines the order that fields are identified and is called once for a save and a second time for a load.
The Defintion() method accepts a EODBDescriptor, and you in turn call one of it's four methods:
Structure is used to define attribute level data and what it defines must be a known type, as defined by the "Type Support" section above. Using the Structure is generally fairly easy, it consists of a GETTER function that returns data and a SETTER action that accepts some data. The type of data has to match:
public void Structure<T>(Func<T> aGetter, Action<T> aSetter) { .. }
It's convenient to use LAMBDA and an anonymous delegate for the getter and setter:
aDescriptor.Structure(()=>CharacterPosition,W=>CharacterPosition = W);
This calls structure, specifying the value to write out, and how to get the data back into the object during a read.
This call implies that you can only read and write from local variables. But this is not the case. Any attribute value is completely acceptable, such as:
aDescriptor.Structure( ()=>gameObject.transform.position , A=>gameObject.transform.position = A);
aDescriptor.Structure( ()=>gameObject.transform.rotation , A=>gameObject.transform.rotation = A);
aDescriptor.Structure( ()=>gameObject.GetComponent<MeshRenderer>().materials[0].color
, A=>gameObject.GetComponent<MeshRenderer>().materials[0].color = A);
The only requirement for what is acceptable are the types that you declare in EODBTypeIO.Initialize().
While I am using LAMBDA here, you can put in your own methods, to do more complicated conversions:
aDescriptor.Structure(GetData,SetData);
Index is used to define attribute level data that can be included into a index. As mentioned earlier, you can index any kind of data that is a known type and that type doesn't have to be the same as the data it's referencing. Ultimately you are using an index to search for data, therefore, whatever format simplifies you subsequent searching is acceptable:
public void Index<T>(string aIndexIdentifier, Func<T> aData) { ... }
Any value you want can be placed into the index, and whatever value you specify becomes associated with your OID. For example, this is valid:
aDescriptor.Index("name", ()=>Last + ", " +First);
This places the concatenation of "Last, First" into the index "name" is associated with whatever your object's OID is.
In some cases, you may want to explicitly remove an object from the index. You do so with either of these two calls:
aDescriptor.Index("name" , ()=>null);
aDescriptor.Index("name" , null);
This method allows you to read and write ICollection<T>, enabling saving and loading of:
See the Tutorials for videos on using Collections with List<> and Dictionary<>
Note: See Arrays below for saving Array[] objects.
Collections work on individual elements of the supplied subject and function similarly to IEODBClass.Definition, but with extra parameters. During a save, your aDefintion Action<> is called multiple times for each instance in the Subject. During a load, you add values to aSubject.
public void Collection<T>(Action<EODBDescriptor, ICollection<T>, T> aDefinition,
ICollection<T> aSubject) {
During a read a default value is passed to aSubject, but is ignored. Instead, you describe the attributes you need to read for one T and exit. aDefinition is called each time it encounters an entry in the file. You don't need to worry about how many you get, just your logic needs to handle the loading of several individual ones.
For every main type that EODB supports, it also supports an Array equivalent of that type.
You need to use Array<T>() only when saving and loading arrays when working with object instances. Working with primitive types and structures (i.e. Vector3) is much easier, because you can use .Structure().
You need to use Array<T>() to save and load arrays of class instances, however, to set the context as to where to add the object, you need an additional method to identify how to add a new T. This extra method provides an additional feature, it allows you to construct an object that contains the context in which the Add takes place. This allows you to handle multi-level hierarchies. Note: This add method also exists on an alternate form of the Collection() method.
public void Definition(EODBDescriptor aDescriptor) {
aDescriptor.Structure(()=>Name,A=>Name=A);
aDescriptor.Structure(()=>Position,A=>Position=A);
if ( aDescriptor.isLoading )
GamerFoods = new Food[0];
aDescriptor.Array(FoodDefinition,GamerFoods,AddFood);
}
private void FoodDefinition(EODBDescriptor aDescriptor, Food[] aGamerFoods, Food aFood, Action<Food> aAddFood) {
string THE_NAME = "";
int THE_COUNT = 0;
aDescriptor.Structure(()=>aFood.InventoryName,NAME=>THE_NAME = NAME);
aDescriptor.Structure(()=>aFood.InventoryCount,COUNT=>THE_COUNT = COUNT);
if (aDescriptor.isLoading) {
aAddFood(new Food(THE_NAME, THE_COUNT));
}
}
public void AddFood(Food aFood) {
Array.Resize(ref GamerFoods, GamerFoods.Length + 1);
GamerFoods[GamerFoods.Length - 1] = aFood;
} 
1.8.9.1