Maiden-Relay

About Maiden-Relay

The Maiden relay allows transparent event relaying across a network of cores. Optimally no other module should have to be aware that the relay is in effect. However, there must always be some source of authority that first establishes the connections and issues the proper subscriptions so that events get transmitted at all.

How To

In order to set your core up for accepting relay connections, simply instantiate a relay consumer and add it to the core. If you simply want to be able to send and receive events without allowing anyone to actually connect to you, set either :host or :port to NIL. Using connect you can then connect to a remote relay. With subscribe you can subscribe to events that happen on a remote relay and have them be transmitted over to your currently local core. Note that you can construct a network that bridges over several cores, but your network must never contain cycles. If a cycle is introduced, the system will immediately enter a message passing infinite loop until the cycle is broken.

Also note that remote clients get recreated locally through virtual-client instances. The purpose of this is such that the client field in events can be properly tagged and such that you can respond to events without having to be aware that the event came from a different core. Additionally you can access slots on the client transparently, but accessor functions will need an additional method. You can even construct relaying methods such that a call of a method with a virtual client will be executed on the core local to the virtualised client. To do this, see define-virtual-client-method.

Internals

Connection Establishment

When a relay initiates a connection with a new remote, the clients at both ends follow the following procedure:

Issue a connection-initiated event
Send the following message: (:id relay-id maiden-version)
Send a network update containing the local relay in its new slot.

Plain lists are treated specially upon receiving. They indicate relay internal communication and are handled as follows:

:id Check the version against the local one, signal a client-version-mismatch warning if there's a mismatch. Then set the received id as the remote of the client.
:timeout Signal a client-timeout-error error.
:close Call close-connection on the client.
:ping Send a (:pong) message back.
:pong Do nothing.

Otherwise the following handling steps are taken depending on the type of message:

network-update Call update on the relay with the remote and a reconstructed update to propagate the cĥange. Reconstructing the update through make-network-update has the effect that the hop count of the update is increased by one.
subscription-update Call update on the relay with the remote and the update.
client-event Call relay on the relay with the remote and relay.
transport Call relay on the relay with the transport target and relay.
Otherwise signal an unknown-message-warning.

Connection reestablishment in case of disruptions or timeouts are handled by the respective Maiden base client mixins.

Network Persistence

The network state gets persisted through network-updates. An update can contain a list of bad links that should be removed or new links that should get recognised. Each entry in the new list is of the form (hops client-id client-name), and each entry in the bad list is of the form destination-id. For each update that gets processed we must know its next stop origin, which we know by looking at which client received it. For bad links we then look at each client it lists. If it's a virtual-client, remove each link in its link table that contains this origin. If there are no more links left, remove the client entirely. For new links if the client does not exist, it is created and added to the cores of the relay. Then each link's corresponding virtual client gets a new entry with the hop count and origin saved.

This way each particular virtual client knows which directly connected relay to relay the message to next to reach itself, but it does not know how the full network path. Since each link is annotated with a hopcount it should also be able to choose the path that requires the least hops. This is not necessarily the fastest, but we assume that the latency addition of a hop outweighs the badness of a potentially slow link.

Message Relaying

A relay instance has a global handler that acts on all events. Unless the origin of the event is 'relay It checks if the event is a client event and the client is a virtual client. If that is the case, it relays the event. Otherwise, if the event matches a subscription, it wraps it in a transport and relays it.

When a relay client receives a client-event or a transport, it is too sent to relay as described in the Connection Establishment section above.

relay then does the following depending on what the target is:

core The event is issued onto the core, but making sure to change the origin to 'relay and fixing the event-loop slot.
consumer The event is issued onto whatever core can be found in the relay's list that contains the client.
virtual-client The closest link in the virtual-client that still exists is looked up and the message is sent to the corresponding client.
(eql T) The message is sent to all clients on the relay.

Message Serialisation

See maiden-serialize.

Copyright

maiden-relay is licensed under the Artistic license. © Nicolas Hafner <shinmera@tymoon.eu> . This library can be obtained on https://github.com/Shinmera/maiden.

Package Index

MAIDEN-RELAY (ORG.SHIRAKUMO.MAIDEN.CLIENTS.RELAY)

class
CONNECT
```
 
```

class

`GENERIC-CALL-EVENT`

Event to represent a generic call request.

When executed, the requested form is evaluated.

See EXECUTE-INSTRUCTION
See FORM

class

`NETWORK-UPDATE`

An update to notify of a change in the network.

The update contains a list of new connections that should
be added, and a list of bad connections that should be
removed.

See NEW
See BAD
See MAKE-NETWORK-UPDATE

class

`QUERY-EVENT`

Superclass for all query events in the system.

Queries are events that represent both a change request and
a request for a result to be obtained about the change. They
often represent a "full" function call.

It is identified so that it and its response event counter-
piece can be found.

See DEFINE-QUERY
See DEEDS:IDENTIFIED-EVENT
See INSTRUCTION-EVENT

class

`RELAY`

This agent represents a relay node.

It is a TCP server that accepts remote connections from
other relay nodes. It manages the updates that need to be
sent and distributed over the PTP relay network. It also
takes care of capturing events that are relevant for other
nodes and sends them out to the interested parties.

There can only be one relay node per core, as otherwise
the messaging and networking infrastructure could much
too easily become confused.

Currently there is no support for loops in the PTP network.
If a cycle appears, it will spawn and endless loop of
updates that will clog the network.

In order for clients to be visible to other nodes in the
network the relay must notify all other nodes of additions
and removals of clients on its core. It must then also
create "virtual" representations of the clients on the
other cores on its local core. Without this, it would not
be possible for the system to completely create a
transparent layer that allows agnostic communication with
clients on remote cores.

See SUBSCRIPTIONS
See MY-SUBSCRIPTIONS
See VIRTUAL-CLIENT
See MAIDEN-NETWORKING:TCP-SERVER
See MAIDEN:AGENT

class

`RELAY-CLIENT`

This class represents a remote client that was connected to this system via the relay.

See REMOTE
See MAIDEN-NETWORKING:TCP-SERVER-CLIENT
See MAIDEN-NETWORKING:TIMEOUT-CLIENT

class

`RELAY-INSTRUCTION-EVENT`

Superclass for all relay events that represent an instruction to be performed on the remote end.

The event can be "executed" via the EXECUTE-
INSTRUCTION function.

See EXECUTE-INSTRUCTION
See MAIDEN:INSTRUCTION-EVENT

class

`RESPONSE-EVENT`

Generic response event class.

This event is used to deliver the response payload of a
query-event. Its IDentity must be the same as that of the
query-event that prompted it.

See DEEDS:IDENTIFIED-EVENT
See DEEDS:PAYLOAD-EVENT
See PASSIVE-EVENT

class

`SLOT-BOUNDP-EVENT`

Event to represent a slot bound test request.

When executed, it tests whether the slot is bound and sends
the result back over the network.

See EXECUTE-INSTRUCTION
See SLOT-EVENT

class

`SLOT-EVENT`

Superclass for all events relating to slot operations on CLOS objects.

The idea of these events is that they would carry operations
that should be performed on an object that exists on a remote
system and is not "actually there" on the local system.

See SLOT
See OBJECT
See DATA-RESPONSE-EVENT

class

`SLOT-MAKUNBOUND-EVENT`

Event to represent a slot unbinding request.

When executed, the object's slot is made unbound.

See EXECUTE-INSTRUCTION
See SLOT-EVENT

class

`SLOT-SETF-EVENT`

Event to represent a slot value set request.

When executed, the object's slot value is set to the
requested value.

See EXECUTE-INSTRUCTION
See SLOT-EVENT
See VALUE

class

`SLOT-VALUE-EVENT`

Event to represent a slot value read request.

When executed, the object's slot value is read and sent back
over the network.

See EXECUTE-INSTRUCTION
See SLOT-EVENT

class

`SUBSCRIBE`

This event starts the subscription process for a certain event-type.

After a subscription has been sent out, events matching
it on remote cores will be relayed to this core and
will thus be visible.

See SUBSCRIBE

class

`SUBSCRIPTION`

An update that describes a desire to have certain kinds of events relayed.

See SUBSCRIPTION-UPDATE
See EVENT-TYPE
See FILTER

class

`SUBSCRIPTION-UPDATE`

Base class for updates relating to the subscription of events.

See TARGET
See SUBSCRIBER
See SUBSCRIPTION
See UNSUBSCRIPTION

class

`TRANSPORT`

An update that carries an event to a destination.

See EVENT
See TARGET
See MAKE-TRANSPORT

class

`UNSUBSCRIBE`

This event starts the unsubscription process for a certain subscription.

See SUBSCRIPTION
See MY-SUBSCRIPTIONS
See UNSUBSCRIBE

class

`UNSUBSCRIPTION`

An update that describes a desire to no longer have certain kinds of events relayed.

See SUBSCRIPTION-UPDATE

class

`VIRTUAL-CLIENT`

This client is a virtual representation of a client instance that exists on a remote core.

Operations on the core will be relayed over the network.
You will not be able to use accessors on the virtual-
client, even if it does "stand in" for a certain type
of client that should allow you to use those. However,
accessing slot values should work just fine, albeit they
may be slow to return, since they need to be routed over
the network.

See LINKS
See MAIDEN:CLIENT

condition

`CARRIER-CONDITION`

Superclass for all conditions related to the carrying of the updates across the relay network.

See RELAY-CONDITION
See MESSAGE

condition

`CLIENT-VERSION-MISMATCH`

Condition signalled when the versions of the two relay endpoints are not compatible and a connection cannot be established.

See RELAY-CONDITION
See REMOTE-VERSION

condition

`NO-RELAY-TARGET-SPECIFIED`

Condition signalled when the target of an update to be relayed got lost or was not specified properly.

See CARRIER-CONDITION

condition

`RELAY-CONDITION`

Superclass for all conditions related to the relay system.

See MAIDEN:CLIENT-CONDITION

condition

`RELAY-LINK-NOT-FOUND`

Condition signalled when a link between two relays was attempted to be used, but appears to be missing.

See CARRIER-CONDITION
See TARGET-CONDITION

condition

`RELAY-ROUTE-NOT-FOUND`

Condition signalled when no possible route to the specified target can be found.

See CARRIER-CONDITION
See TARGET-CONDITION

condition

`TARGET-CONDITION`

Superclass for all conditions that relate to a certain target relay.

See RELAY-CONDITION
See TARGET

accessor (

`BAD`

OBJECT)

Accessor to the list of bad links to be removed.

This is a list of link names.

See NETWORK-UPDATE

accessor (

`EVENT-TYPE`

OBJECT)

Accessor to the type of events that the subscription update should make a subscription for.

See SUBSCRIPTION

accessor (

`FILTER`

OBJECT)

Accessor to the filter description that limits the events that the subscription update should make a subscription for.

See SUBSCRIPTION

accessor (

`LINKS`

OBJECT)

Accessor to the list of links the actual client can be reached through.

The list should be sorted by proximity of the hops.

See VIRTUAL-CLIENT

accessor (

`MY-SUBSCRIPTIONS`

OBJECT)

Accessor to the list of SUBSCRIBE objects this node itself generated.

See RELAY
See SUBSCRIBE

accessor (

`NEW`

OBJECT)

Accessor to the list of new links to be registered.

This is a list of lists, where each inner list has the
following elements:
- HOP-COUNT    The number of hops necessary to reach the
               destination.
- DESTINATION  The ID of the core of the destination.
- NAME         The name of this particular link.

See NETWORK-UPDATE

accessor (

`REMOTE`

OBJECT)

Accessor to the identifier of the remote core that is connected over this relay-client.

See RELAY-CLIENT

accessor (

`RESPONSE-EVENT`

OBJECT)

Accessor to the response event the handler captures.

accessor (

`SUBSCRIBER`

OBJECT)

Accessor to the ID describing the remote that has sent the subscription update.

See SUBSCRIPTION-UPDATE

accessor (

`SUBSCRIPTIONS`

OBJECT)

Accessor to the list of SUBSCRIBE objects to keep track of.

See RELAY
See SUBSCRIBE

accessor (

`TARGET`

CONDITION)

The ID of the target core that the object is related to.

See TARGET-CONDITION
See SUBSCRIPTION-UPDATE
See TRANSPORT

function (

`CONNECT`

CORE772 &KEY (HOST "127.0.0.1") (PORT 9486))

This function initiates a connection to a remote relay node.

See CONNECT

function (

`SUBSCRIBE`

CORE821 EVENT-TYPE FILTER &OPTIONAL (TARGET T))

This function starts the subscription process for a certain event-type.

See SUBSCRIBE

function (

`UNSUBSCRIBE`

CORE843 SUBSCRIPTION)

This event starts the unsubscription process for a certain subscription.

See SUBSCRIPTION
See MY-SUBSCRIPTIONS
See UNSUBSCRIBE

generic (

`FORM`

OBJECT)

Reader for the form that should be evaluated for the call event

See GENERIC-CALL-EVENT

generic (

`MAKE-NETWORK-UPDATE`

NEW BAD)

This function takes care of creating an appropriate network update to send out to the relays directly connected to this core.

It takes a NEW and BAD argument, which can be the
respective lists directly, COREs, CONSUMERS, or already
existing NETWORK-UPDATES.

See NETWORK-UPDATE

generic (

`MAKE-TRANSPORT`

EVENT TARGET)

Create a transport object to carry the event to the target.

See TRANSPORT

generic (

`MAKE-VIRTUAL-CLIENT`

TARGET &KEY NAME LINKS)

Create a virtual-client instance for the given target.

See VIRTUAL-CLIENT

generic (
MESSAGE
CONDITION)
```
 
```

generic (

`OBJECT`

OBJECT)

Reader for the object that the event relates to.

See SLOT-EVENT

generic (

`PROCESS`

MESSAGE TCP-CLIENT)

This is called to handle an update from the remote.

The update may be an object of any type, depending on what the
client deals with. The client will likely want to convert the
update object into an event of some kind and issue that event
onto its cores.

If the data can be parsed, but the client doesn't understand it,
a condition of type UNKNOWN-DATA-WARNING is signalled.

See HANDLE-CONNECTION
See UNKNOWN-DATA-WARNING

generic (

`RELAY`

MESSAGE TARGET RELAY)

Relay a message object to the target over the relay.

This will take care of, if possible, potentially
encapsulating the object and sending it off to its
destination over the PTP network.

See UPDATE
See RELAY

generic (

`REMOTE-VERSION`

CONDITION)

Reader for the version of the remote relay system.

generic (
RESPONSE
OBJECT)
```
 
```

generic (

`ROUTABLE-P`

TARGET NETWORK)

Returns whether the given target seems to be reachable, or the given event seems to be able to reach its destination from the current relay.

See RELAY

generic (

`SLOT`

OBJECT)

Reader for the slot that the event relates to.

See SLOT-EVENT

generic (

`SOURCE`

OBJECT)

Reader for the source of the event. This should be an identifier for who requested the instruction.

See DATA-RESPONSE-EVENT

generic (

`UPDATE`

RELAY UPDATE-SOURCE UPDATE)

Process an update from a source on a relay.

Updates should only be processed on the relay that it
is intended for. This function is also responsible for
updating the internal representation of the network
as well as for delivering the update event to the core.

See RELAY
See NETWORK-UPDATE
See SUBSCRIPTION-UPDATE

macro (
DEFINE-VIRTUAL-CLIENT-METHOD
&REST ARGS)
```
 
```