1.0.0

A dialogue system language implementation.

About Speechless

This is a system implementing an advanced dialogue system that is capable of complex dialogue flow including choice trees and conditional branching. Speechless was first developed for the Kandria game, and has since been separated and made public in the hopes that it may find use elsewhere or inspire other developers to build similar systems.

Speechless is based on the Markless document standard for its syntax and makes use of Markless' ability to be extended to add additional constructs useful for dialogue systems.

Speechless can compile dialogue from its base textual form into an efficient instruction set, which is then executed when the game is run. Execution of the dialogue is completely engine-agnostic, and only requires some simple integration with a client protocol to run.

Thanks to Markless' extensibility, Speechless can also be further extended to include additional syntax and constructs that may be useful for your particular game.

User Manual

This section covers the syntax and behaviour of Speechless dialogue. With any luck, after reading this you'll be able to write your own dialogue in Speechless.

Speech

Let's jump in with the classic:

| Hello world!

This piece should just show the text Hello world!. However, there's a bit of a problem: we don't know who's saying this! In order to annotate which character is speaking, we can add a source line:

~ Hans
| Hello world!

Now we know that this is a character called Hans speaking the line. What the character's name should actually be depends on how your game engine organises things. For now we're assuming that we can just use the character's own name.

Often you'll also want to inform the system about how the character is feeling to accentuate the text and give it some more gravitas.

~ Hans
| (:excited)Hello world!

Depending on your game this emote might correspond to the character changing their expression on screen as the text scrolls. For further flavour we can also use Markless' markup to change the text style.

~ Hans
| (:excited)//Hello// **world**!

In Markless itself this would mean italic and bold, respectively, though the actual interpretation of it is up to the current game engine, and might have different effects like text shaking, wobbling, and so forth. Important to know here is only that there are several ways to markup text out of the box:

Markup can also be nested, //like **so**!//.

Finally, you can accentuate text by inserting pauses with en (--) and em (---) dashes. This can be quite useful to control the pacing of the text scroll to be more natural.

~ Hans
| Helloooo-- world!

After a line of text, Speechless will insert a prompt for the player to continue so that there's no risk of the text scrolling by too fast for the player to read it.

~ Hans
| Hey
| (:concerned)Are you still with me?
| (:frantic)Hang in there, Chris!

That should cover things for speech. Let's move on to more advanced stuff.

Player Choice

Speechless allows you to make branching dialogue by presenting the player with a set of options. These options can even be gated behind checks. A basic choice is presented through a simple unordered list.

~ Hans
| Are you getting this?
- Yes
- No

This will show the options Yes and No to the user, though no matter what they end up picking, neither of them actually do anything. Let's change that.

~ Hans
| Are you getting this?
- Yes
  | Nice, well done!
- No
  | Sorry to hear that. What are you having trouble with?

The consequences of a choice follow after the choice's text, within the same list item. You can insert as many things as you want into the same list item as long as the item actually continues (no empty lines).

~ Hans
| Are you getting this?
- Yes
  ~ Hannah
  | See, I told you, they'd get it no problem!
- No
  ~ Hannah
  | Ah what, really? Come on!

You can insert as many choices as you want, though obviously for game design reasons it's probably not wise to present too many at once.

Choices can also be nested to create dialogue trees.

~ Hans
| Are you getting this?
- Yes
  | Alright, are you ready for the next lesson?
  - Let's go
    | Wow, you're really blazing through this!
  - No, not quite yet
    | Ok, let's continue another time then
- No
  | What's bothering you?
  - I just need more time
    | Ok, no problem, let's try again another time then
  - I'm hopelessly confused
    | Give yourself some time then, come back to it later!
  - I hate this
    | Sorry to hear that. You don't have to continue if you don't want to!

As you might imagine, this can quickly get hard to see through, though. One way to make it a bit easier to read is to destructure the choices with labels and jumps.

Labels and Jumps

In order to control flow and to help you section your dialogue, Speechless includes constructs to give points within the dialogue a name and to jump to such a name from another point in the dialogue. Creating a label is simple enough:

> start
| How are ya?

This associates the label start with the speech How are ya?, meaning once you jump to start it will start showing the text. Jumping to a label is also simple enough. For fun, let's recreate the classic BASIC loop.

> 10
| HELLO WORLD!
< 10

This will infinitely repeat, creating a truly never ending experience. All jokes aside, jumps and labels can come in handy when writing branching dialogue, since it allows us to create "empty choices" and to flatten the tree.

> start
| Are you sure you want to give me a million dollars?
- Yes!
  | Wow, thanks so much, you're so generous!
- No
  | I implore you to reconsider.
  < start

This will cause the dialogue to keep looping back to the question if the player chooses the No option, effectively creating an empty choice as found in some games.

Similarly we can flatten choice trees by sectioning consequences out to labels. Transforming the above choice we get this:

~ Hans
| Are you getting this?
- Yes
  < Yes
- No
  < No

> Yes
| Alright, are you ready for the next lesson?
- Let's go
  | Wow, you're really blazing through this!
- No, not quite yet
  | Ok, let's continue another time then

> No
| What's bothering you?
- I just need more time
  | Ok, no problem, let's try again another time then
- I'm hopelessly confused
  | Give yourself some time then, come back to it later!
- I hate this
  | Sorry to hear that. You don't have to continue if you don't want to!

There's one small problem with this though, namely that once the Yes tree completes, it will continue execution and also do the No tree. This is pretty much never what we want, so instead we should be using sections to do this, which is another way of labelling:

~ Hans
| Are you getting this?
- Yes
  < Yes
- No
  < No

# Yes
| Alright, are you ready for the next lesson?
- Let's go
  | Wow, you're really blazing through this!
- No, not quite yet
  | Ok, let's continue another time then

# No
| What's bothering you?
- I just need more time
  | Ok, no problem, let's try again another time then
- I'm hopelessly confused
  | Give yourself some time then, come back to it later!
- I hate this
  | Sorry to hear that. You don't have to continue if you don't want to!

Sections force an end of dialogue, so that if execution should hit upon a section header without it just jumping there, it'll automatically end. If we now want to go back to the main execution flow after either choice completes however, we can use a normal label again:

~ Hans
| Are you getting this?
- Yes
  < Yes
- No
  < No
> Main
| Bye!

# Yes
| Alright, are you ready for the next lesson?
- Let's go
  | Wow, you're really blazing through this!
- No, not quite yet
  | Ok, let's continue another time then
< Main

# No
| What's bothering you?
- I just need more time
  | Ok, no problem, let's try again another time then
- I'm hopelessly confused
  | Give yourself some time then, come back to it later!
- I hate this
  | Sorry to hear that. You don't have to continue if you don't want to!
< Main

Note that all labels – regardless of the syntax used to define them – are global to a piece of dialogue though, so you can't use the same name twice.

Lets move on to another advanced topic, conditions and branching.

Conditions

Sometimes dialogue needs to react to game state. Choices might need to be hidden, or a character might need to react in a different way depending on what the player has done so far. Speechless offers a couple of ways to do this. The most basic way is the conditional block.

? have-water
| ~ Fireman
| | Quick, use the water to put out the fire!

In this dialogue, the Fireman will shout Quick, use the water to put out the fire! if have-water is true.

have-water here is a placeholder for some kind of Lisp expression. This expression can be arbitrary Lisp code, and it is up to the game engine to control how that code is evaluated. Often the engine will add a variety of support functions and variables to help manage and check various parts of the game's state. We won't go into Lisp syntax and semantics here, you'll want to consult another tutorial for that as it is a very complex topic.

The above conditional block can also include additional branches, and simply executes the first branch for which the test passes:

? have-water
| ~ Fireman
| | Quick, use the water to put out the fire!
|? have-hose
| ~ Fireman
| | Blast it!
|?
| ~ Fireman
| | Someone get us some water, stat!

A branch with no condition always passes. The conditional block is thus similar to if/else if/else chains, or cond in Lisp.

Often however the full conditional block is too lengthy and bulky. For quick conditional dialogue pieces, the inline branch can also be used:

~ Fireman
| [have-water Quick, use the water! | Get some water stat!]

Both forms of branching can be used to restrict choices behind a check as well.

- [have-water Here, let me help!|]
  ~ Fireman
  | Then don't just stand there!
- I'll go get some water!
  ~ Fireman
  | And make it quick!

In this case if the have-water check fails, the option would not be available to the player.

One thing to watch out for when using conditional blocks to restrict choices is that since the block and speech both use the same syntax, you have to make sure to interrupt the conditional block before you can continue with speech. You can do this with another block like a source, or by using a clear (not empty!) line:

- ? have-water
  | Here, let me help!
  
  | Then don't just stand there!

Note the two spaces on the seemingly empty line to continue the list item.

Both forms of conditionals can be nested arbitrarily, too, though you cannot nest a conditional block inside an inline conditional.

Evaluation

Finally, often it's desired to cause changes to game state, or include dialogue particular to game state. Since integrating with game state is typically extremely specific to the game and engine being made at the time, Speechless just includes basic constructs to splice a Lisp value into text, and to evaluate arbitrary Lisp code.

While not ideal from a user perspective as it forces you to learn some Lisp, typically engine coders can provide enough shorthand functions for the dialogue system such that dialogue authors don't have to learn a lot of Lisp to get going.

The placeholder {form} evaluates the form and writes whatever the form returns as its value to text. This can be handy when you need to involve dynamic stuff like nicknames in a classic RPG:

| Hi, my name is {(name character)}!

Dialogue also often needs to trigger changes in game state, be that in response to completing a dialogue or in response to a choice the user made. For this the eval instruction can be used.

! eval (spawn 'dog)

Again, which functions and variables are available depends on the game and engine being built. Please consult your internal documentation on that.

Misc

Markless includes a few additional constructs that can be helpful with Speechless, too. For instance, comments can be used to write notes for yourself about what a piece of dialogue is meant to do or in which context it occurs.

; This is a note to myself and won't have any effect!

You can also emit warnings or even errors if there's a problem in the dialogue and you would like to make sure the game can't proceed yet.

! error This isn't ready yet!

Again, consult the Markless documentation for a complete outline of the Markless syntax and semantics.

This should cover everything that's supported by Speechless out of the box. Your engine may add extra constructs and support syntax. To see how to do that, please read on.

Writing a Client

The dialogue system in Speechless is game engine agnostic. This means that you get full control over the user input and display of the dialogue.

Before we can get started on implementing the client, you should know that Speechless is a compiled language. Thus you need to compile the dialogue to an assembly before it can be executed. To do so, you can simply call compile*. If you would like to customise the lexical environment for forms used in the dialogue however, you should also create a subclass of assembly, implement a method on wrap-lexenv, and then pass an instance of that assembly to compile*. All Lisp forms within the dialogue are also compiled down to functions, so no run-time eval or compilation occurs when dialogue is actually executed.

(defclass assembly (dialogue:assembly) ())

(defmethod dialogue:wrap-lexenv (form (_ assembly))
  `(let ((example T))
     ,form))

(defvar *dialogue* (dialogue:compile* "| Hey there!" 'assembly))

Once armed with a compiled assembly, you'll want to execute it. To do so you need a vm instance, and prepare it with run.

(defvar *vm* (make-instance 'dialogue:vm))
(dialogue:run *dialogue* *vm*)

From there on executing dialogue is a matter of repeatedly calling resume on the VM and performing whatever request it returns. To do so you'll want to implement a loop similar to this:

(loop with ip = 0
      for request = (dialogue:resume *vm* ip)
      until (typep request 'dialogue:end-request)
      do (setf ip (handle request)))

Though typically you won't be able to use a synchronous loop like this, since the game will need to render and perform other functions like scrolling the text or waiting for user input before resuming the VM.

In any case, the interactions with the VM are simple – all you need to keep track of is the next instruction pointer to resume with once you're ready to continue execution. Each request that the VM returns to you will have one – or in the case of a user choice several – target instruction pointers to resume with.

If you still need help implementing it, you can check the Kandria sources for a fully featured reference implementation.

If you have trouble with dialogue execution and it doesn't behave as you expect, you can try debugging it by passing the assembly to disassemble. This behaves similar to cl:disassemble and will print out the assembled instructions in their execution sequence.

Extension

If desired you can add your own syntax constructs to Speechless. To do so you will need to implement a corresponding syntax tree component, a directive to parse it, an instruction to execute it, and possibly a request to expose whatever client functionality you need.

Before adding new syntax, you should familiarise yourself with the Markless standard terminology, and the cl-markless implementation. As an example here though we'll assume you want to implement a new "singular line directive" – something that has a prefix at the beginning of the line and consumes the whole line, like ~ .

We're going to add some syntax to let you spawn some objects in the game. The syntax should look like this:

+ enemy :at 'camp

First we need to define a component class corresponding to our new syntax.

(defclass spawn (org.shirakumo.markless.components:block-component)
  ((arguments :initarg :argmunts :accessor arguments)))

Next we need to define a directive that will parse the syntax to the new component type.

(defclass spawn-directive (org.shirakumo.markless:singular-line-directive) ())

(defmethod org.shirakumo.markless:prefix ((_ spawn-directive))
  #("+" " "))

(defmethod org.shirakumo.markless:begin ((_ spawn-directive) parser line cursor)
  (let ((arguments (loop while (< cursor (length line))
                         collect (multiple-value-bind (value next) (read-from-string line NIL NIL :start cursor)
                                   (setf cursor next)
                                   value)))
        (component (make-instance 'spawn :arguments arguments)))
    (org.shirakumo.markless:commit _ component parser))
  (length line))

Next we need to add our directive onto the list of used directives for the dialogue.

(pushnew 'spawn-directive org.shirakumo.fraf.speechless.syntax:*default-directives*)

Now that that's done we can already parse our new syntax!

(dialogue:parse "+ test")

To compile this we could re-use an existing instruction like eval, but just to be complete we'll define separate instruction types and requests.

(defclass spawn-instruction (dialogue:instruction)
  ((type :initarg :type :accessor type)
   (argfun :initarg :argfun :accessor argfun)))

(dialogue:define-simple-walker spawn spawn-instruction
  :type (first (arguments spawn))
  :argfun (compile NIL `(lambda () ,(dialogue:wrap-lexenv `(list ,@(rest (arguments spawn))) dialogue:assembly))))

Also see walk and emit in case you ever have more complex needs for compilation. The reason we compile the rest of the arguments here is that we probably want them to be evaluated, rather than literals.

Now we can compile our dialogue to an assembly:

(dialogue:compile* "+ test")

Finally we need to define the execution of our instruction. Again, a new request is likely not necessary here, but we'll do it just for completeness of the tutorial.

(defclass spawn-request (dialogue:target-request)
  ((type :initarg :type :accessor type)
   (initargs :initarg :initargs :accessor initargs)))

(defmethod dialogue:execute ((instruction spawn-instruction) (vm dialogue:vm) ip)
  (dialogue:suspend vm (make-instance 'spawn-request :type (type instruction) :initargs (funcall (argfun instruction)) :target (1+ ip))))

When we execute the instruction we call the argument function to evaluate the arguments to a list. We then return that list fully prepared to the client, so that all they need to do now is create the new instance as requested and spawn it into the game.

We also include a target instruction pointer, which usually is just the instruction immediately following.

And that's it. Now you can fully compile and execute your new dialogue syntax!

For more advanced instruction compilation, optimisation passes, and other stuff, please see the source code.

System Information

1.0.0
Nicolas Hafner
zlib
https://shirakumo.github.io/speechless
https://github.com/shirakumo/speechless.git

Definition Index