This book is a work in progress, comments are welcome to: johno(at)johno(dot)se

Juice

Introduction

The primary goal of Juice was to provide a standardised file format capable of expressing heirarchical key/value data. It was important that the file format was plain text, both human readable and easily editable in standard ANSI text editors.

As a data source, Juice consists of two parts, the declaration part (.fruit files, analogous to .h files) and the definition part (.juice files, analogous to .cpp files). Once the data was defined, programmers could manipulated this data via a C++ library that supported load time validation of the data.

An interesting point is that Juice is in essence much the same thing as XML, however it was created in ignorance at a time when all things "webby" were shunned by the programmers at Massive Entertainment.

Data Declaration (.fruit)

A programmer would typically create a .fruit file that declares custom datastructures based on language primitives by using reserved keywords, as below. Note that C++ style comments are supported. All reserved keywords are BOLD in Juice.

//BEGIN example.fruit

//custom TYPE, analogous to C++ enum
TYPE ObjectType
{
    ROBOT
    TANK
    HELICOPTER
}

//custom CLASSes, analogous to C++ classes
CLASS Position
{
    DECIMAL myX
    DECIMAL myY
    DECIMAL myZ
}

CLASS ScreenPosition
{
    NUMBER myX
    NUMBER myY
}

CLASS Object
{
    ObjectType myType
    Position myPosition
    NUMBER myId
    TEXT myName
    LOCTEXT myLocalizedComment
}

CLASS PlaySound
{
    FILE myFile
}

CLASS ShowPicture
{
    FILE myFile
    ScreenPosition myPosition
}

SCRIPTCLASS EventList
{
    PlaySound
    ShowPicture
}

Note that in the above example, the built-in types DECIMAL, NUMBER, TEXT, FILE, and LOCTEXT are used. These are built in basic types and are all implemented using a string (std::string or whatever you prefer) as the actual data holding object. This means that there is no technical difference between DECIMAL, NUMBER, TEXT, FILE, and LOCTEXT.

The main reason for the differentiation in the typename of the various keywords is that certain tools use this information to present the user with various means of editing the fields. Also, it helps programmers and designers remember what kinds of data should be entered into a given field. Also, the dependency tracking software depends upon FILE extensively to resolve project dependencies (i.e. always use FILE when you are actually referencing a file).

The use of LOCTEXT denotes that this field should be localized (support wide / Unicode characters). This is used by Ice.

SCRIPTCLASS is a special kind of list. There is no clear analogy in C++, but one can see it as a sort of template parameterized linked list, but with the capability of storing objects of different types. This can be used for event scripts (hence the name).

Data Definition (.juice)

To actual use custom data structures, a a .juice file will be created that INCLUDEs the previously created custom data declarations, in the same way that one would do in C++.

//BEGIN example.juice

INCLUDE example.fruit

Object Tank
{
    myType TANK
    myPosition
    {
        myX 5
        myY 3
        myZ 7
    }

    myId 0
    myName "John Bob"
    myLocalizedComment "How are you doing?"
}

EventList AppEvents
{
    PlaySound IntroSound
    {
        myFile sound/bullet/artillery1_1.wav
    }

    ShowPicture SplashScreen
    {
        myFile ui/backgrounds/loading_01/loading_01_02.tga
        myPosition
        {
            myX 0
            myY 0
        }
    }
}

Note that Juice itself does not support any form of wide / Unicode characters, only ANSI. Ice however has the capability of supporting UTF-8.

In practice, .juice was rarely created by hand in a text editor, but rather using the JuiceMaker.

Usage (C++)

The Juice library offers a C++ interface to manipulate Juice data.

//BEGIN example1.cpp

#include "juicecompiler.h"
#include "juicenamespace.h"
#include "juicetype.h"

bool LoadJuice()
{
    JuiceCompiler c;
    JuiceNameSpace ns;
    JuiceType* tank;
    JuiceType* pos;

    //load the file
    if(!c.Compile(ns, "example.juice"))
        return false;

    //access a top level instance
    tank = ns.GetInstanceForName("Tank");
    if(!tank)
        return false;

    //access a member instance
    pos = tank->GetMember("myPosition");

    //assign a value to a member of an instance
    pos->GetMember("myX")->Assign(60.f);

    //access the values of an instance's members
    fprintf(stderr, "%f %f %f",
        pos->GetMemberValue("myX"), pos->GetMemberValue("myY"), pos->GetMemberValue("myZ"));

    //save changes
    if(!ns.Persist("example.juice"))
        return false;

    return true;
}

In the above example, the file example.juice is compiled to the JuiceNameSpace object ns on the stack. Next, the object Tank is accessed, and some member manipulation is done, changing a value, and then displaying some values. Lastly, the namespace is saved back to disk to the file example.juice again.

The basic operations most application used were variations of GetValue(), GetMember(), and Assign(). These existed in various permutations and overloads to facilitate ease of use.

Technically, the Juice library was based on the Composite design pattern, in that the main interface seen by clients was the JuiceType interface, which represented any instance in a namespace. The concrete class in each case would vary, but the interface of JuiceType was designed in such a way that all capabilities of TEXT, NUMBER, DECIMAL, FILE, LOCTEXT, custom CLASSes and SCRIPTCLASSes were exposed (a typical caveat of the Composite pattern).

Uses

Juice as a run-time memory format and persistence solution

Josephine was the first project to use Juice to define all data values for all content, including event scripts, and also as a solution for the "scary" persistence problem of save-games. Historically, Massive had seen the problem of save games as a very large one, and had thus not implemented it in Ground Control. In a shopping-mall sim like Josephine the game was essentially a single long sequence of events that all affected the future, we realised the need for persistence. For this reason, we decided to use Juice as both a persistence format (this was trivial as Juice supported both loading and saving easily), but also as the run-time memory format.

This in itself turned out to be quite dubious for many reasons. To begin with, performance turned out to be an issue, because obviously using strings in heirarchical linked list structures was not as speedy as using native C++ types (like ints and floats). Towards release, issues with overly long load times were resolved by creating the first incarnation of Ice, which later evolved into the data-format solution for Ground Control 2.

Secondly, and admittedly more nasty, Juice implicitly supported reflection, and having access to reflection tends to result in slightly "magical" architectures. Just so in Josephine, where messages in the form of JuiceType instances were published/broadcast to any number of subscribed listeners via abstract channels. To severely understate, this made the system extremely hard to debug and understand.

While the ideas of overly abstract, reflection-based event-driven system did not survive, Ground Control 2's mission editor XED was also implemented using Juice as both persistence and run-time memory formats. It was the contention of the programmer involved that this was a win for the project.

Also, as in Ground Control the concept of mission/level scripts was used in both Josephine and Ground Control II, in both these cases using Juice as the data format.

Problems

Implicit and invisible dependencies between .fruit/.juice and C++ code

In practice, the client code was required to implicitly know about the type of object it was accessing in terms of what member names were etc. There was as a result an implicit and invisible dependency between client code and the datastructures declared in each .fruit file. The main issue with this was that access errors could not be detected by a C++ compiler, only at run-time (often as a crash), and it ultimately fell to the programmer to make sure that all accesses were valid.

A classic example of an error that would crash at runtime is calling GetMemberValue() with a member name that did not exist in the given JuiceType instance. This was implemented as GetMember("name")->GetValue(), and if GetMember("name") returned NULL (denoting that the named member did not exist), then the code would produce an access violation at run-time.

Note that these kinds of problems could also occur when .fruit files were changed without the client code being changed. For this reason, .fruit files should really be considered as part of the code itself (due to the tight coupling between the .fruit file and the client code), and be placed under source control with the rest of the code. In practice (during Josephine and Ground Control 2), .fruit and .juice files were NOT under source control, as the were viewed as data.

Heirarchical and code-centric data definitions

Data structures were most often defined in .fruit to mirror the required run-time structures, which were most often heirarchical. Many times the structures created were logically confusing, but tended to be bent to work withing the constraints of Juice (this also tends to be true of the use of XML). Because the .fruit files were created by programmers in order to server the need of code created by the same programmers (ease of access in C++ was often a high priority), it was often the case that the designers who were tasked with entering the data into these structures were at a loss.

Summary

The most important driving factor of the creation of Juice was a desire to minimize the need of creating custom editors for the various data needs of games. At the time, we believed firmly in the the old "gui's are hard" truth, and by creating a file format that could server all our custom data needs, we believed we could avoid much custom gui work (MFC being the preferred gui solution at the time).

Over time, plug in architectures evolved in JuiceMaker in order to facilitate more specific and advanced editors.

Since then, research into IMGUI and various alternative persistence solutions (not to mention the above noted problems inherent to Juice) have led my away from using text based data files.