Framsticks network client/server specification 

A few clients are available for the Framsticks server.

Framsticks Network Server is a powerful simulation engine, with all capabilities of the Framsticks command-line interface. Its application depends only on network clients, which can use the server for research, remote simulation, presentation of evolutionary processes and phenomena, distributed evolution, modeling of ecosystems and migration, real-time interaction in mixed realities, and much more.

1. Assumptions

  • connecting to the server via telnet gives the interface of the command-line Framsticks version
  • network client will call server macros, which allows protocol independence from implementation and FramScript objects
  • server allows standardized access for any objects, their attributes and functions
  • client may present them in a standard way (tables, buttons)
  • for some classes of objects (genotypes, creatures, world), specialized interface may be used by clients (displaying geometry, neural network, etc.)
  • safety, access rights: login/password may be introduced in the future. For example, the command "login [user] [passwd]" will be required for some commands to work.

2. Commands, data and messages flow

The server is able to support different protocols for different clients. Client must send the required protocol version using the "version" directive. The "ok" answer means that the protocol is supported and can be used, otherwise "error" is returned. (For compatibility, the "client_v1" directive is also supported). Interactive clients could also support the "telnet" mode (which means just passing user-typed commands to the server), which exposes many internal details of the simulator and can be used even if the higher level negotiation failed.

Each object can be accessed using its path. Path sections are separated by the / char.

  • The root object is /
  • If the attribute "a" of an object with path "p" contains another object, then path "p/a" is allowed
  • If the attribute "l" of an object with path "p" is of type "list", then you can refer elements of this list using path
    • p/l/n (n is a 0-based list element index)
    • p/l/id (id is an unique ID within the list)
  • The unique ID allows to refer to the same element independently from changes which may occur on the list. Some lists may not support these unique IDs, which can be determined by checking for an extended type definition of the list property:
    • "l ClassName" – list of ClassName objects
    • "l ClassName idname" – as above, the elements can be addressed by the "idname" value
A command is a single line of text (which is a call of a server macro, usually with parameters) ended with \n or \r\n. Command arguments are separated by spaces. When an argument itself needs to contain a space, enclose it with quotation marks " around the argument "like this". Just like in popular programming languages, characters like backslash \, quotation marks ", newline, and tabulation that are placed inside argument values have special backslash codes: \\, \", \n, \t. Backslash has another special purpose (unrelated to its quoting function) when it is put as the last character in the line – this enables entering multiline string as a single command, which is a general Framsticks command-line behavior. This can unfortunately interfere with the network protocol, so make sure there are no trailing backslashes in your input to avoid surprises. One simple way to avoid this is to add a space character after the backslash, when your last argument ends with a backslash. Another one is to enclose arguments (at least the last one) in quotes.

After this command is executed, it is acknowledged by the message 

ok [id] [optional_text]
If the command could not be completed, the acknowledgement is 
error [id] [optional_error_description]

Note: The [id] argument (in commands and in responses) is only present if the "request_id" feature is enabled (see the use command).

Further, server event sources send event objects to clients. A client needs to register for an event source to make the server send this event's objects. There are two event handler types. The type 0 handler emits events immediately, type 1 handler sends events in response to the "send" or "sendall" function calls. Both handlers use the following format:

event [eventpath] [eventname]
file
....(event data)...
eof
For some commands, the client needs to send a file (formatted in a special way, as most Framsticks files). For some commands, the server sends a file (formatted as above). The files are sent before the command execution acknowledgement "ok". This acknowledgement means that the data transmission related to the command is finished.
The file sending mode is initialized by the sender, which sends the directive: 
file [id] [filename] [file_description]
The parameters are optional.
From now on, data is treated as file contents, until the end-of-file marker is received. This marker is exactly three chars in a single line: 
eof
If the "eof" exists in the file sent, it must be "quoted" as \eof (\eof must be quoted as \\eof etc.)
The server tells that it needs a file to complete some command by sending request 
needfile [id] [suggested_name] [file_description]

3. Commands

In examples below, the following underline convention is used:
data sent by the client to the server
data sent by the server to the client

use

use [feature_name]
Feature negotiation – returns "ok" if the requested feature is supported. The following features can be negotiated:
  • request_id – enables the first [id] argument in set, get, call, reg, info commands. The server responds with the same [id] in ok, error and file messages. This is meant as a synchronization aid, especially for pipelining-enabled clients.
  • needfile_id – this feature is only effective together with request_id enabled. When both features are enabled, each needfile from the server is also sent with the extra [id] as its first argument, and file responses must repeat the same [id] in their first argument.
  • groupnames – enables the additional group:NAME_OF_THE_GROUP field in property definitions ("property:" objects) reported by the info command. Group membership is purely presentational, designed for a meaningful ordering of large sets of properties. Not all properties have group names.
  • worldmap_SCRIPTNAME or worldmap_SCRIPTNAME_SCRIPTARGUMENT – requests an alternate worldmap generation script. This influences what data is returned in the geometry field of /simulator/world and allows certain advanced clients to provide their own custom scripts without disrupting regular clients. The default behavior is to use the "WorldMap_faces.script" (this file can be found in the "scripts" subdirectory), which returns a simple polygonal model of the world, described in Appendix A.

Example message exchange with both request_id and needfile_id enabled: 

call MYID /simulator netload
needfile MYID "*.expt" "Experiment"
file MYID
...file content...
eof
ok MYID

version

version [number]
Returns "ok" if the requested protocol version is supported or "error" otherwise.
The current protocol version is 5. You don't need to know about the other versions – just use the use command to declare what features are required.

info

info [id] [path]
Returns information related to the object referred as "path". The result is a file with definitions of properties (objects of the "property" class, just as can be found in many Framsticks files, like *.expdef, *.style, *.neuro, etc.)

The file may contain the name of the object's class, given as the "id" property of the object "class". This class name can be used by the client to check if the object is of known type, for example, to use specialized methods of visualization and interaction.

If the last element of the path is a property of the list type, then the result is the definition of properties of objects which the list contains. Analogously, the event class definition is returned from the event source query.
The classname is usually contained in the "l" and "e" type description ("l ClassName" and "e ClassName") so you only need to issue the info command on "l" and "e" if you need more details.

Example:

info /
file
class:
id:Server

property:
id:simulator
name:Simulator
type:o Simulator

property:
id:cli
name:Commandline interface
type:o CLI

property:
id:motd
name:Message of the day
type:s

eof
ok

set

Two variants: 
set [id] [path]
(requires sending a file)
Will set properties of the object according to the file sent.
set [id] [path] [property] [value]
Will set (only one) value – thus there is no need to send a file.

Examples:

set /someobject
needfile
file
field:value
field2:value2
eof
ok

set /badobject
error "invalid path for set"

set /someobject field value
ok

get

get [id] [path] [attributes]
(returns a file)

Sends a file with attributes of the given object. Optionally, you can provide a list of names (comma-separated). If there is no list, all attributes will be sent.

For attributes of type list, the number of list elements is returned as the attribute value. If the path refers to a property of type list, the resulting file will contain data for all objects on the list.

Starting with the protocol version 3, the path can contain multiple branches. This allows for querying multiple objects in a single get request. Results are identical with what could be achieved without branching (using multiple queries) except for the data consistency: simulator state can change between subsequent gets, but not between evaluating multiple branches in a single get.

  • Branching is denoted with parentheses and commas (just like in f1 Framsticks genotypes), e.g. /path(/branch1,/branch2,.../branchN)
  • List enumeration: a special character + can be used as the list element. It automatically creates branches for all elements in the list, e.g. the following path refers to creatures lists in all groups: get /simulator/populations/groups/+/creatures
  • Individual branches can be queried for different attributes. This is defined in the curly brackets immediately after the branch, e.g. get /simulator({expdef,running},/populations/groups/+/creatures/+{name,energy})
  • Responses for each branch variant are returned in individual files. Each file is named after the branch.
  • Limitation: the branched path will always return "ok", even if some (or all) path variants are invalid. The returned file names can be useful for determining what went wrong in such cases.

Examples:

  • Get a single attribute: 
    get / motd
file
Server:
motd:Welcome to the Framsticks server.

eof
ok
  • Get a single attribute of type list (i.e. getting the number of list elements): 
    get / somelist
file
Server:
lista:2
eof
ok
  • Get all attributes for all objects on the list: 
    get /somelist
file
object:
id:id1
property:123

object:
id:id2
property:456

eof
ok
  • Get only the id attribute for all objects on the list: 
    get /somelist id
file
object:
id:id1

object:
id:id2

eof
ok
  • Get only the selected property of the given objects on the list – selecting by the consecutive number 
    get /somelist/0 property
file
object:
property:123
eof
  • Get all attributes of the chosen object on a list – selecting by the unique id 
    get /somelist/id1
ok
file
object:
id:id1
property:123
eof
ok
  • Get the simulator status, names, energy and basic physical properties of all creatures in all populations (branched path example) 
    get /simulator({expdef,running},/populations/groups/+/creatures/+({name,energy},/(parts{x,y,z},joints{p1,p2})))

file /simulator
Simulator:
expdef:standard
running:0

eof
file /simulator/populations/groups/0/creatures/0
Creature:
name:Uwuwit Si
energy:5000

eof
file /simulator/populations/groups/0/creatures/1
Creature:
name:Itinih Sih
energy:10000

eof
file /simulator/populations/groups/0/creatures/0/parts
Part:
x:0
y:0
z:0

Part:
x:1
y:0
z:0

eof
file /simulator/populations/groups/0/creatures/1/parts
Part:
x:0
y:0
z:0

Part:
x:1
y:0
z:0

Part:
x:2
y:0
z:0

eof
file /simulator/populations/groups/0/creatures/0/joints
Joint:
p1:0
p2:1

eof
file /simulator/populations/groups/0/creatures/1/joints
Joint:
p1:0
p2:1

Joint:
p1:1
p2:2

eof
ok
  • Valid branched path that does not return a file (because the list is empty) 
    get /simulator/genepools/groups/0/genotypes/+

ok
  • Similar but not identical results: branching vs. list contents 
    get /simulator/populations/groups/0/creatures/+ name

file /simulator/populations/groups/0/creatures/0
Creature:
name:Uwuwit Si

eof
file /simulator/populations/groups/0/creatures/1
Creature:
name:Itinih Sih

eof
ok

    get /simulator/populations/groups/0/creatures name

file
Creature:
name:Uwuwit Si

Creature:
name:Itinih Sih

eof
ok
  • Shared field list can be used with branches (although it is not recommended) – nonexistent fields will be silently ignored. Here, the "expdef" field is only found in "Simulator:" and "name" field in "Creature:" 
    get /simulator(,/populations/groups/+/creatures) expdef,name
file /simulator
Simulator:
expdef:standard

eof
file /simulator/populations/groups/0/creatures
Creature:
name:Uwuwit Si

Creature:
name:Itinih Sih

eof
file /simulator/populations/groups/1/creatures
eof
ok

call

call [id] [path] [function] [parameter1] [parameter1] [parameter3] ...
(may require files)
(may return files)

Calls a function of an object referred as path. The behavior of this function and its parameters depend on the object. The information about function parameters is available in the property description. The information about whether some files are involved will be indicated as usual (file, needfile).
Function arguments are space-delimited, use "..." (quotes) to pass arguments containing space characters.
The network interface provides access to all functions available in FramScript, even if their call'ing is not useful. This may change in the future – some commands may be restricted for network use, e.g. by some flag.

The return value is returned inside the "Result" object (see examples).

Examples:

call /simulator netsave
file experiment.expt "current experiment state"
...contents of the file sent by the server...
eof
ok

call /simulator netload
needfile "*.expt" "Experiment"
file
...contents of the file sent to the server...
eof
ok

call /simulator/genepools worst
file
Result:
value:-1
eof
ok

reg

reg [id] [path] [type] [event_id]

Registers for a specified event. Event handlers are visible in the objects hierarchy as the fields of type "e".
reg returns the following message on successful registration:

ok registered_event_path
The returned path can be used to modify the handler properties and unregister from the event source (it is a regular network object).
[type] and [event_id] are optional (will replace the default values when used).

Registered events can be managed through the /cli/events collection (you get the actual path from the reg confirmation, don't rely on the /cli/events constant). Each event handler has the following properties:

  • id – unique id. Can be used to access the event in the events collection.
  • name – event name, initially set to the event path from which it was created.
  • type – handling type: 0=immediate ("server push"), 1=store and return on demand ("client pull"), 2=disabled.
  • count – event count (useful for "store" and "disabled" handlers).
"sendone" and "sendall" function calls are used for requesting stored events from type 1 handlers.
Stored events are returned before the "ok" confirmation for the "call" command so that you always know when the event stream is finished. 
call /cli/events/e0 sendall
....(multiple events)...
ok

Each event sent to a client contains the header (event event_id event_name) and the event file. The file contains event-specific data sent in the regular object format. The info command can be used to read the event class details (see the ListChange examples).

Examples:

reg /cli/messages 1
ok /cli/events/e0
get /cli/events
file
Event:
id:e0
name:/cli/messages
type:1
count:0

eof
ok
call /simulator/print "test"
ok
- doesn't output anything because of the type=1 handling 
call /cli/events/e0 sendall
event e0 /cli/messages
file
Msg:
class:Script
function:Message
message:test
level:0

eof
ok

4. Some details

File format and object contents

A file may contain any number of objects of various classes. Objects contain fields with names and values. 
ClassName:
fieldname:value
fieldname2:value2
...etc

  <-( empty line ends the object contents )

ClassName2:
somefieldname:somevalue
...etc

If a field value contains \n, then the whole value is embedded in ~ signs:

fieldname:~
a multi-line value here, there may be
many lines of
text~
If there is a ~ sign within the field value, it must be quoted as \~

Objects of class "property"

Descriptions of object fields are used in many script files in Framsticks, and contain 5 sections:
  • id – internal field name (for scripts, and for paths in the network protocol described here)
  • name – user-friendly name
  • type – data type. The first char determines the base type. The following chars can provide more information. Base types are "d"=integer, "f"=floating point, "s"=string, "o"=object, "x"=universal type, "p"=function/procedure, "l"=a list of elements, "e"=event source. In practice, the client should focus on:
    • d/f/s – these fields can be displayed as numbers/texts in a standard way
    • o – you can further explore fields of the object type (just add to path). The type definition can include the object class name, like this: "o ClassName".
    • l – list also publishes sub-objects, in a different way than "o".
    • p – procedure/function. Can be called by "call".
    • e – event source. The type descriptor can include the event class name, like this: "e ClassName". Clients can assume the predefined event content and semantics from the well known class name, e.g. "Msg" = general human readable message (former "msg"), "ListChange" = list contents change notification. The actual class definition can be read by the "info" command.
    Read the full property type specification to implement a conforming client.

  • flags – additional information in a single number. The most important bits:
    • 1 – readonly
    • 16 – user should not modify this field
    • 32 – user should not see this field
    Read the full flags specification to implement a conforming client.

  • help – additional description (for a tooltip, or documentation)

Objects of the "class" class

These are small quasi-objects which have only one attribute named "id", which contains the class name. It is used for sending object description.

ListChange event

A client can register for this event to get notifications when the list contents changes. The event source named "LISTNAME_changed" with type "e ListChange" is the event source for the list "LISTNAME". The following example gets the event class description from the server: 
info /simulator/populations/groups/0/creatures_changed
file
class:
id:ListChange

property:
id:type
name:type
type:d 0 2 ~Add~Remove~Modify

property:
id:pos
name:position
type:d
help:modify position<0 means 'all items modified'

eof
ok
The "pos" field contains 0-based index of the added, removed or modified list item. For the "Modify" event, a negative position can be specified, which means "all items were modified".

Example: (monitoring of the creatures list)

reg /simulator/populations/groups/0/creatures_changed
ok  /cli/events/e0
call /simulator/populations/groups/0 createFromString X
event e0 /simulator/populations/groups/0/creatures_changed
file
ListChange:
type:0
pos:0

eof

file
Result:
value:<Creature object at 0x8128ee8>
eof
ok

5. Server implementation remarks

  • in some cases, quoting ~ signs does not work for file transfers.
  • file-based set works only for "true" object fields, net.ini pseudo fields are ignored.
  • some critical fields (e.g. /cli/script) influence the internal FramScript implementation and should not be set.
  • the only lists capable of the (optional) unique-id access are the event list, genotypes lists and creatures list. The unique field's name is "uid" (genotype and creature) or "id" (event).

6. Some examples

Command line arguments

To test the server, you can also use the command-line interface without the -n switch. Useful options: -e disables the default error message printing, -s disable "newshell" (console handling) under linux (required when piping commands from files).

frams -e -s -inet.ini "info /" -q
frams -e -s -inet.ini "get /simulator/world" -q
frams -e -s -inet.ini "info /simulator/populations/groups/0/creatures" -q
frams -e -s -inet.ini "call /simulator/populations/groups/0 createFromString X[N,1:1][G]X" "get /simulator/populations/groups/0/creatures/0/mechparts x,y,z" -q

Sample commands sent to server

  //create a simple creature for the subsequent queries
call /simulator/populations/groups/0 createFromString XX[N,1:1][G]

  //get populations count
get /simulator/populations groups

  //get population names
get /simulator/populations/groups name

  //get gene pools names
get /simulator/genepools/groups name

  //get all populations properties
get /simulator/populations/groups

  //get first population count
get /simulator/populations/groups/0 creatures

  //get all creature properties from the first population
get /simulator/populations/groups/0/creatures

  //get some properties from the first creature
get /simulator/populations/groups/0/creatures/0 name,genotype,info,parts,joints,neurons

  //get parts details (static model) from the first creature
get /simulator/populations/groups/0/creatures/0/parts

  //get joint connections from the first creature
get /simulator/populations/groups/0/creatures/0/joints p1,p2

  //get the first mech parts details (current simulation data) from the first creature
get /simulator/populations/groups/0/creatures/0/mechparts/0

  //get all mech part coordinates from the first creature
get /simulator/populations/groups/0/creatures/0/mechparts x,y,z

  //get current neuron states from the first creature
get /simulator/populations/groups/0/creatures/0/neurons state

Controlling neurons and muscles

To control a neuron/muscle, set its "hold" property to 1.
To set (force) the neuron value, use the "currState" field.
Send to server: 
call /simulator/populations/groups/0 createFromString XXX[N,1:1][G]
set /simulator/populations/groups/0/creatures/0/neurons/0 hold 1
set /simulator/populations/groups/0/creatures/0/neurons/0 currState 0.44
get /simulator/populations/groups/0/creatures/0/neurons/0

Determining running state

Determining if the simulation was started or stopped.
  • Check the "running" property of the simulator object ("polling" approach) 
    get /simulator running
file
Simulator:
running:0

eof
ok
  • Register for the running_change event (event-driven approach) 
    reg /simulator/running_changed
ok  /cli/events/e0
    Events can be delivered via polling or pushed by the server, as usual. Whenever the simulation is started, the client will receive 
    event e0 /simulator/running_changed
file
ValueChange:
value:1

eof
    ...and "value:0" when the simulation is stopped.
    The running_changed event is sent as the "ValueChange" object. The client can get the object definition using the following command (as it can be done with all available events and fields): 
    info /simulator/running_changed
file
class:
id:ValueChange

property:
id:value
name:value
type:d

eof
ok

7. Further development

  • access control, rights

Appendix A. Default format of the 'geometry' property – string representation of the world

The format is a subset of the well known Wavefront .obj.

Clients can request different kind of data by providing a custom script and invoking use worldmap_SCRIPTNAME (see also Feature negotiation).

A sample with comments:


v 0 0 0
v 1 0 0
v 1 0 1
.......
f 1 2 3
f 3 2 4
f 3 4 5 6
......

Lines starting with 'v' are vertices (3D coordinates).
This vertex has index 1
2
3

Lines starting with 'f' are faces, every face is built from 3 or 4 vertices, referred by vertex index.
This is a triangle.
This is a quad.

Appendix B. Hierarchy of objects – a sample generated for server version 2.9

  /
  +-cli
  +-simulator
    +-world
    +-genepools
    | +-groups
    |   +-0
    |   | +-genotypes
    |   |   +-0
    |   |   +-1
    |   |   +-2
    |   ...
    +-populations
      +-groups
        +-0
        | +-creatures
        |   +-0
        |   | +-parts
        |   | | +-0
        |   | | +-1
        |   | +-joints
        |   | | +-0
        |   | | +-1
        |   | +-neurodefs
        |   | | +-0
        |   | | +-1
        |   | +-mechparts
        |   | | +-0
        |   | | +-1
        |   | +-mechjoints
        |   | | +-0
        |   | | +-1
        |   | +-neurons
        |   |   +-0
        |   |   +-1
        |   +-1
        |   | ...
        |   +-2
        |     ...
        +-1
        | +-creatures
        |   +-0
        |   | ...
        |   +-1
        |   ...
        ...