This is the specification for the Lichat chat protocol version 2.
The Lichat protocol defines the following basic data type hierarchy:
number
integer
time An integer representing time as the number of seconds since 1900.1.1 1:0:0 UTC.
float
id
symbol
keyword A symbol whose package is keyword.
boolean
null The symbol NIL from the lichat package.
true The symbol T from the lichat package.
list
(list type) A list with each element being of type type.
string
username See §2.2.1
channelname See §2.4.4
password See §2.3.1
object
The wire format is based on UTF-8 character streams on which objects are serialised in a secure, simplified s-expression format. The format is as follows:
UPDATE ::= OBJECT NULL
OBJECT ::= '(' WHITE* SYMBOL (WHITE+ SYMBOL WHITE+ EXPR)* WHITE* ')'
EXPR ::= STRING | LIST | SYMBOL | NUMBER
STRING ::= '"' ('\' ANY | !('"' | NULL))* '"'
LIST ::= '(' WHITE* (EXPR (WHITE+ EXPR)*)? WHITE* ')'
SYMBOL ::= KEYWORD | NAME | NAME ':' NAME
KEYWORD ::= ':' NAME
NUMBER ::= '0..9'+ ( '.' '0..9'*)? | '.' '0..9'*
NAME ::= (('\' ANY) | !(TERMINAL | NULL))+
TERMINAL ::= (':' | ' ' | '"' | '.' | '(' | ')')
WHITE ::= U+0009 | U+000A | U+000B | U+000C | U+000D | U+0020
NULL ::= U+0000
ANY ::= !NULLA symbol is an object with a "name" that is home to a "package". A "package" is a simple collection of symbols. Symbols are uniquely identified by their name and package, and no two symbols with the same name may exist in the same package. The package and symbol names are case-insensitive, and two names are considered the same if they match after both have been transformed to lower case.
This protocol specifies two core packages: lichat and keyword. Symbols that come from the lichat package must be printed without the package name prefix. Symbols from the keyword package must be printed by their name only prefixed by a :. Every other symbol must be prefixed by the symbol's package's name followed by a :.
When a symbol is read, it is checked whether it exists in the corresponding package laid out by the previous rules. If it does not exist, it may be substituted for a placeholder symbol. Servers must take special care not to keep symbol objects they don't know around, to avoid letting clients overload the server's memory with inexistent symbols.
Special care must be taken when objects are read from the wire. An error must be generated if the object is malformed by either a non-symbol in the first place of its list, imbalanced key/value pairs in the tail, or non-keywords in the key position. An error must also be generated if the symbol at the first position does not name a class that is a subclass of object. Finally, any field whose value is specified as nil on the wire can be equated with being unset, and vice versa when printing an object if an optional field is unset or nil, it may be omitted from the wire representation.
Null characters (U+0000) must not appear anywhere within a wireable. If a string were to contain null characters, they must be filtered out. If a symbol were to contain null characters, the message may not be put to the wire.
All object types are specified in lichat.sexpr in a machine-readable format based on the above wire format. The same format should also be used by extension providers to define their extensions to the protocol. The definitions can be parsed by parsing the file into a sequence of EXPRs, each being further parsed according to the following DEFINITION rule:
DEFINITION ::= PACKAGE | OBJECT | EXTENSION
PACKAGE ::= (define-package PACKAGE-NAME)
PACKAGE-NAME ::= STRING
OBJECT ::= (define-object CLASS-NAME (SUPERCLASS*) SLOT*)
CLASS-NAME ::= SYMBOL
SUPERCLASS ::= SYMBOL
SLOT ::= (SLOT-NAME TYPE OPTIONAL?)
FIELD-NAME ::= SYMBOL
TYPE ::= number | integer | time | float | id | symbol | keyword
| boolean | null | true | list | (list TYPE) | string
| username | channelname | password | object | T
OPTIONAL ::= :optional
EXTENSION ::= (define-extension EXTENSION-NAME EXTENSION-DEFINITION*)
EXTENSION-NAME ::= STRING
EXTENSION-DEFINITION ::= OBJECT | OBJECT-EXTENSION
OBJECT-EXTENSION ::= (define-object-extension CLASS-NAME (SUPERCLASS*) SLOT*)The meaning of each definition is as follows:
package Introduces a new, known package with the PACKAGE-NAME as its name.
object Introduces a new object type using CLASS-NAME as its name. Each SUPERCLASS must name an object type that was previously introduced, and whose fields and behaviour should be inherited. Each SLOT defines a slot that the object holds in addition to the inherited ones. If the slot definition includes the :optional keyword, the slot may be omitted when serialising to the wire. If :optional is not included, the slot must be serialised. When translating from the wire, an omitted :optional slot should be initialised to an empty value. An omitted non-:optional slot must result in a malformed-update error.
extension Introduces a protocol extension of the given EXTENSION-NAME. Its body includes new definitions that need to be added if the extension is to be supported.
object-extension Changes an existing object type of CLASS-NAME by either introducing additional SUPERCLASSes, or introducing additional SLOT definitions on the object. Note that in order to stay backwards compatible, every slot specified via an object-extension must be :optional.
The server must keep track of a number of objects that are related to the current state of the chat system. The client may also keep track of some of these objects for its own convenience.
Each client is connected to the server through a connection object. Each connection in turn is tied to a user object. A user may have up to an implementation-dependant number of connections at the same time.
users represent participants on the chat network. A user has a globally unique name and a number of connections that can act as the user. Each user can be active in a number of channels, the maximal number of which is implementation-dependant. A user must always inhabit the primary channel. A user may have a profile object associated with it. When such a profile exists, the user is considered to be "registered." The server itself must also have an associated user object, the name of which is up to the specific server instance.
A user's name must be between 1 and 32 characters long, where each character must be from the Unicode general categories Letter, Mark, Number, Punctuation, and Symbol, or be a Space (U+0020). The name must not begin or end with
with Spaces (U+0020), nor may two Spaces be consecutive. Two user names are considered the same if they are the same length and each code point matches case-insensitively.
The profile primarily exists to allow end-users to log in to a user through a password and thus secure the username from being taken by others. A profile has a maximal lifetime. If the user associated with the profile has not been used for longer than the profile's lifetime, the profile is deleted.
A profile's password must be at least 6 characters long. It may contain any kind of character that is not Null (U+0000).
channels represent communication channels for users over which they can send messages to each other. A channel has a set of permission rules that constrain what kind of updates may be performed on the channel by whom. There are three types of channels that only differ in their naming scheme and their permissions.
Exactly one of these must exist on any server, and it must be named the same as the server's user. All users that are currently connected to the server must inhabit this channel. The channel may not be used for sending messages by anyone except for system administrators or the server itself. The primary channel is also used for updates that are "channel-less," to check them for permissions.
Anonymous channels must have a random name that is prefixed with an @. Their permissions must prevent users that are not already part of the channel from sending join, channels, users, or any other kind of update to it, thus essentially making it invisible safe for specially invited users.
Any other channel is considered a "regular channel".
The names of channels are constrained in the same way as user names. See §2.2.1.
A permission rule specifies the restrictions of an update type on who is allowed to perform the update on the channel. The structure is as follows:
RULE ::= (type EXPR)
EXPR ::= t | nil | EXCLUDE | INCLUDE
EXCLUDE ::= (- username*)
INCLUDE ::= (+ username*)
Where type is the name of an update class, and username is the name of a user object. t is the symbol T and indicates "anyone". nil is the symbol NIL and indicates "no one". The INCLUDE expression only allows users whose names are listed to perform the action. The EXCLUDE expression allows anyone except users whose names are listed to perform the action. The expression t is thus equivalent to (-) and the expression nil is equivalent to (+).
The default permissions for the primary channel should look like this, where :registrant should be replaced with the server user's username:
(capabilities T)
(channels T)
(connect T)
(create T)
(disconnect T)
(grant (+ :registrant))
(join T)
(kick (+ :registrant))
(leave NIL)
(message (+ :registrant))
(permissions (+ :registrant))
(ping T)
(pong T)
(pull NIL)
(register T)
(search T)
(server-info (+ :registrant))
(user-info T)
(users T)
The default permissions for the primary channel should look like this, where :registrant should be replaced with the channel creator user's username:
(capabilities T)
(channels T)
(deny (+ :registrant))
(grant (+ :registrant))
(join T)
(kick (+ :registrant))
(leave T)
(message T)
(permissions (+ :registrant))
(pull T)
(users T)
The default permissions for the primary channel should look like this, where :registrant should be replaced with the channel creator user's username:
(capabilities T)
(channels NIL)
(deny NIL)
(grant NIL)
(join NIL)
(kick (+ :registrant))
(leave T)
(message T)
(permissions NIL)
(pull T)
(users T)
The client and the server communicate through update objects over a connection. Each such object that is issued from the client must contain a unique id. This is important as the ID is reused by the server in order to communicate replies. The client can then compare the ID of the incoming updates to find the response to an earlier request, as responses may be reordered or delayed. The server does not check the ID in any way– uniqueness and avoidance of clashing is the responsibility of the client. Each update should also contain a clock slot that specifies the time of sending. This is used to calculate latency and potential connection problems. If no clock is specified, the server must substitute the current time. Finally, each update may contain a from slot to identify the sending user. If the from slot is not given, the server automatically substitutes the known username for the connection the update is coming from.
When an update is sent to a channel, it is distributed to all the users currently in the channel. When an update is sent to a user, it is distributed to all the connections of the user. When an update is sent to a connection, it is serialised to the wire according to the above wire format specification. The actual underlying mechanism that transfers the characters of the wire format to the remote host is implementation-dependant.
At the end of each update there has to be a single null character (U+0000). This character can be used to distinguish individual updates on the wire and may serve as a marker to attempt and stabilise the stream in case of malformed updates or other problems that might occur on the lower level.
If an update cannot be performed for some reason, the server will respond with a failure update. If the client's update was successfully parsed, this failure must be a subclass of update-failure, which will reference the failed update's id in the update-id slot, so that the client can reconstruct which of the requests failed.
In addition to the normal reply of an update – whether successful or failed – the server may also send an update that is a subclass of warning. This update must be from the primary user, and must reference the original update in its update-id slot. The client may use the warning to inform the user or adjust future behaviour in response to the server's notice.
Note that a warning may only be issued if the request is processed otherwise. The server must not only reply with a warning. The server further must reply with the warning before replying with the response to the original update.
After the connection between a client and a server has been established through some implementation-dependant means, the client must send a connect update. The update will attempt to register the user on the server, as follows:
If the server cannot sustain more connections, a too-many-connections update is returned and the connection is closed.
If the update's version denotes a version that is not compatible to the version of the protocol on the server, an incompatible-version update is returned and the connection is closed.
If the update's from field is missing or NIL, the server substitutes a random name that is not currently used by any other user or registered profile.
If the update's from field contains an invalid name, a bad-name update is returned and the connection is closed.
If the update does not contain a password, and the from field denotes a username that is already taken by an active user or a registered user, an username-taken update is returned and the connection is closed.
If the update does contain a password, and the from field denotes a username that is not registered, a no-such-profile update is returned and the connection is closed.
If the update does contain a password, and the from field denotes a username that is registered, but whose password does not match the given one, an invalid-password update is returned and the connection is closed.
If the server cannot sustain more connections for the requested user, a too-many-connections update is returned and the connection is closed.
A user corresponding in name to the from field is created if it does not yet exist.
The connection is tied to its corresponding user object.
The server responds with a connect update of the same id as the one the client sent. The from field must correspond to the user's actual name.
If the user already existed, the server responds with join updates for each of the channels the user is currently inhabiting, with the primary channel always being the first.
If the user did not already exist, it is joined to the primary channel.
The server sends a message update for the primary channel from the primary user. The message contents are up to the server, and should typically be some kind of welcome banner.
Should the user send a connect update after already having completed the connection handshake above, the server must drop the update and respond with an already-connected update.
If the clock of an update diverges too much, the server may respond with a clock-skewed update and correct the timestamp. If the skew varies a lot, the server may drop the connection after replying with a connection-unstable update.
The server must receive an update on a connection within at least a certain implementation-dependant interval that must be larger than 100 seconds. If this does not happen, the server may assume a disconnection and drop the client after replying with a connection-unstable update. If the server does not receive an update from the client within an interval of up to 60 seconds, the server must send a ping update to the client, to which the client must respond with a pong update. This is to ensure the stability of the connection.
If the client sends too many updates in too short a time interval, the server may start dropping updates, as long as it responds with a too-many-updates update when it starts doing so. This throttling may be sustained for an implementation-dependant length of time. The client might send occasional ping requests to figure out if the throttling has been lifted. The server may also close the connection if it deems the flooding too severe.
Instead of a hard-throttle as described in the previous paragraph, the server may also implement a soft throttle by queueing updates and processing them after a delay. Should the client become throttled in this way, the server must send an updates-throttled warning once updates begin to be queued.
A connection may be closed either due to a disconnect update request from the client, or due to problems on the server side. When the connection is closed, the server must act as follows:
The server responds with a disconnect update, if it still can.
The underlying connection between the client and the server is closed.
The connection object is removed from the associated user object.
If the user does not have any remaining connections, the user leaves all channels it inhabited.
The exceptional situation being during connection establishment. If the server decides to close the connection then, it may do so without responding with a disconnect update and may immediately close the underlying connection.
An update is always checked as follows:
If the update is not at all recognisable and cannot be parsed, a malformed-update update is sent back and the request is dropped.
If the update is too long (contains too many characters), a update-too-long update is sent back and the request is dropped.
If the class of the update is not known or not a subclass of update, an invalid-update update is sent back and the request is dropped.
If the from, channel, or target fields contain an invalid name, a bad-name update is sent back and the request is dropped.
If the from field does not match the name known to the server by the user associated to the connection, a username-mismatch update is sent back and the request is dropped.
If the channel field denotes a channel that does not exist, but must, a no-such-channel update is sent back and the request is dropped.
If the target field denotes a user that does not exist, a no-such-user update is sent back and the request is dropped.
If the update is an operation that is not permitted on its target channel, or the primary channel if no target channel is applicable, an insufficient-permissions update is sent back and the request is dropped.
When a user sends a register update, the server must act as follows:
If the server disagrees with the attempted registration, a registration-rejected update is sent back and the request is dropped.
If the profile does not yet exist, it is created.
The password of the profile associated to the user is changed to match the one from the update.
The profile must stay live until at least 30 days after the user associated with the profile has existed on the server.
The server responds by sending back the original register update.
Note that the server does not need to store the password verbatim, and is instead advised to only store and compare a hash of it.
Since a channel has only two bits of information associated with it, the management of channels is rather simple.
Creating a new channel happens with the create update:
The update is checked for permissions by the primary channel.
If a channel of the channel name in the update already exists, the server responds with a channelname-taken update and drops the request.
If the user already inhabits the maximum amount of channels, the server responds with a too-many-channels update and drops the request.
If the channel field is NIL, an anonymous channel is created, otherwise a regular channel is created.
The user is automatically joined to the channel.
The server responds with a join update to the user with the id being the same as the id of the create update.
The channel's permissions can be viewed or changed with the permissions update, if the channel allows you to do so.
The permissions for the channel are updated with the ones specified in the update's permissions field as follows:
1. For each rule in the specified permissions set in the update:
2. If the rule should be malformed or unacceptable, the server responds with a invalid-permissions update and disregards the rule.
3. Otherwise, set the rule with the same type in the channel's rule set to the given rule.
The server responds with a permissions update with the permissions field set to the full permissions set of the channel, and the id being the same as the id of the update the user sent.
See §2.5 for an explanation of the proper syntax of the permissions. Note that the server may reduce the set of rules to a simpler set that is semantically equivalent.
As a shortcut, permissions can also be managed with the grant and deny updates, which change an individual rule. When a server receives a grant update, it must update the corresponding rule as follows:
If the rule is T, nothing is done.
If the rule is NIL, it is changed to (+ user).
If the rule is an exclusion mask, user is removed from the mask.
If the rule is an inclusion mask, user is added to the mask if they are not already on it.
When the server receives a deny update, it must update the corresponding rule as follows:
If the rule is T, it is changed to (- user).
If the rule is NIL, nothing is done.
If the rule is an exclusion mask, user is added to the mask if they are not already on it.
If the rule is an inclusion mask, user is removed from the mask.
After processing either a grant or deny update successfully, the update is sent back to the user.
A user can interact with a channel in several ways.
Joining a channel happens with the join update, after which the server acts as follows:
If the user is already in the named channel, an already-in-channel update is sent back and the request is dropped.
If the user already inhabits the maximum amount of channels, the server responds with a too-many-channels update and drops the request.
The user is added to the channel's list of users.
The user's join update is distributed to all users in the channel.
Leaving a channel again happens with the leave update, after which the server acts as follows:
If the user is not in the named channel, a not-in-channel update is sent back and the request is dropped.
The user's leave update is distributed to all users in the channel.
The user is removed from the channel's list of users.
Another user can be pulled into the channel by the pull update, after which the server acts as follows:
If the user is not in the named channel, a not-in-channel update is sent back and the request is dropped.
If the target user is already in the named channel, an already-in-channel update is sent back and the request is dropped.
If the target already inhabits the maximum amount of channels, the server responds with a too-many-channels update and drops the request.
The target user is added to the channel's list of users.
A join update for the target user with the same id as the pull update is distributed to all users in the channel.
Another user can be kicked from a channel by the kick update, after which the server acts as follows:
If the user is not in the named channel, a not-in-channel update is sent back and the request is dropped.
If the target user is not in the named channel, a not-in-channel update is sent back and the request is dropped.
The user's kick update is distributed to all users in the channel.
A leave update for the target user is distributed to all users in the channel.
The target user is removed from the channel's list of users.
Finally, a user can send a message to all other users in a channel with the message update, after which the server acts as follows:
If the user is not in the named channel, a not-in-channel update is sent back and the request is dropped.
The user's message update is distributed to all users in the channel.
The server can provide a client with several pieces of information about its current state.
Retrieving a list of channels can be done with the channels update, after which the server acts as follows:
For each channel known to the server, the server checks the update against the channel's permissions.
If the permissions allow the update, the channel's name is recorded.
A channels update with the same id as the request is sent back with the channels field set to the list of names of channels that were recorded.
The list of users currently in a channel can be retrieved by the users update, after which the server acts as follows:
If the user is not in the named channel, a not-in-channel update is sent back and the request is dropped.
A list of the users in the channel is recorded.
A users update with the same id as the request is sent back with the users field set to the list of names of users that were recorded.
Information about a particular user can be retrieved by the user-info update, after which the server acts as follows:
If the user is not connected and no profile for the user exists, a no-such-user update is sent back and the request is dropped.
A user-info update with the same id as the request is sent back with the connections field set to the number of connections the user object has associated with it and with the registered field set to T if the user has a profile associated with it.
A user can request a list of updates they are allowed to send to a particular channel using the capabilities update. The server must act as follows:
If the user is not in the named channel, a not-in-channel update is sent back and the request is dropped.
For every update type known to the server, the server checks whether the user is permitted according to the channel's permission rule for the update. If permitted, the update type is added to a list.
A capabilities update with the same id as the request is sent back with the permitted field set to the list of updates gathered in step 2.
Servers may store additional information about a user, such as statistics, IP addresses, and so forth. Such information can be requested through the server-info update. After receiving such an update, the server must act as follows:
If the user is not connected and no profile for the user exists, a no-such-user update is sent back and the request is dropped.
A server-info update with the same id as the request is sent back with the attributes field set to an "association list", and the connections field set to a list of "association lists", one such "association list" per connection the user has. An association list is a list where each element has the following structure:
(SYMBOL EXPR), where:
SYMBOL is a symbol naming the attribute that is being returned.
EXPR is the value for the attribute being returned.
The attributes being returned are dependent on the server and the supported protocol extensions. The set of returned attributes may also differ depending on the user being requested, especially if the user is the server's user.
The following attributes are specified:
channels A list of channel names in which the user resides.
registered-on When the user registered their profile. NIL if they did not.
The following connection attributes are specified:
connected-on The time at which the connection was initiated.
While these attributes are specified in their purpose, a server does not have to return them.
A server or client may provide extensions to the protocol in the following manners:
Additional Update Types
If such an update is sent to a client that does not recognise it, it should be ignored. If such an update is sent to a server that does not recognise it, the server will respond with an invalid-update.
Additional Update Fields
A client or server may extend the existing update classes with additional, optional fields to provide further information or other kinds of behaviour. The server or client is not allowed to introduce additional required fields. When an update with unknown initargs is received, the unknown initargs are to be ignored. An extension may only add existing, well-specified initargs in keyword form (:text, :channel, :target, :update-id). Other additional initargs must be symbols from an extension-owned package.
Additional Constraints
An extension may introduce additional constraints and restrictions on whether existing updates are considered valid.
Additional User Attributes
An extension may specify additional attributes stored in profiles and returned through server-info.
Additional Server Objects
An extension may specify new server objects and new attributes on existing server objects.
Each extension to the protocol should receive a unique name of the form producer-name where producer is an identifier for who wrote up the extension's protocol, and name should be a name for the extension itself. For each extension that a server and client support, they must include the unique name of it as a string in the connect update's extensions list. Each producer also owns a symbol package with the producer's name, in which they may freely specify new symbols.
shirakumo.mess Describes extensions from the Shirakumo collective.
The following are general conventions for server and client implementors. However, they are not mandatory to follow, as they may only make sense for certain types of implementations.
The default port when served over TCP should be 1111, with 1112 being the default for SSL connections.
When specified in URLs, Lichat takes the lichat protocol and follows this general scheme: lichat://host:port/channel#id Meaning the URL path (if given) is used as the name for a channel to join. The URL fragment can be used to specify a specific message id. Note that it is the client's responsibility to ensure that the ID is sufficiently unique so that the URL will link to the correct message. The query part of the URL may be used for client-specific purposes.