Server Setup and Configuration ​

This document explains in depth how to configure your TSF Servers. There are several other components that will be referenced that may be helpful to fully understand the overall TSF grid structure. They will be linked here for reference.

Note: Throughout this documentation, all files may be copied using the 'copy' icon in the upper right corner.

TSF Server ​

On the server side, there are 2 main configurations, the actual TSF Server configuration json file as well as a properties file.

Properties file ​

ExampleSetup.properties

LogFileName=/home/tsfuser/app/conf/global/logback.xml 
 KeyStoreFileName=/home/tsfuser/app/conf/global/TessSrvrKeyKeyStore.p12
 TrustStoreFileName=/home/tsfuser/app/conf/global/TessSrvrTrustKeyStore.p12
 TokenFileNameAndPath=/home/tsfuser/app/conf/global/.sslparams
 KeyPairAlias=serverkey
 JVMArguments=-Xms256m -Xmx2048m
 LibPath=/home/tsfuser/app/lib
 LicenseKey=CUO0A-NAV32-PQ148-0AS3Q-6O51W

In this file, in the first line, LogFileName=, we specify the location of our logback configuration file which determines what our logs will look like, where they will be located, how much the log will hold before overwriting old entries, etc.. You can see an example of this file here. There are also arguments for the JVM that runs the TSF on each server. Generally, one TSF Instance (one single JVM) should be run on each physical server; OR, if a physical server runs multiple docker containers, one TSF Instance can execute in each but we recommend that the cores be allocated to specific docker containers (and not shared across these). KeyStoreFileName, TrustStoreFileName are needed only if you are supporting secure connections. The TokenFileNameAndPath must be present and must contain at least the ACLFilePassword= property (even though the ACL is not really a file). The other 3 parameters in this file: the password of the key and trust stores (and keypair password -- same as keystore password if using a PKCS#12 format) need only be present if you are supporting secure connections. An example of a Token file can be seen here.

TSF Server Configuration file ​

This governs the configuration of our TSF. Each TSF instance will pass in this configuration file at runtime when it starts up, which provides details about how it is configured - the overall TSF grid will be built from these configuration files as the TSF instances come up. There are a few important sections in this configuration file that we will go into detail about below, and a full list of all the parameters and what they do is linked at the bottom of the page. The TSF comes built-in with default values for most of the parameters you will see on that page, and the purpose of the configuration file(s) passed in at runtime is to override any of those configurable parameters.

Database ​

In the server configuration file, the following parameter can be configured:

"dbConfigFilePathAndName": "/home/tsfuser/tsf/conf/hibernate.tsf.mysql.cfg.xml",

This identifies the file with important settings that enable the TSF to connect to an SQL database. Our framework uses hibernate on top of a jdbc driver identified in the .xml file specified. An example of this file can be seen here. You will need to provide the necessary driver and configurations to connect to your SQL-compliant database. We use MySQL, but any such database, e.g., Oracle, PostGRE, etc. will suffice. In this file are the location and port to connect to your SQL database as well as a username and password to access the database. The TSF uses this database for various internal purposes -- among the most important: Keeping an ACL with usernames and passwords that enable a user to connect successfully with the TSF. If using secure connections, this username and password is tied to the DName of the X509 certificate used in the SSL handshake and has to match.

Services and Dynamic Services ​

Another section of the TSF Server configuration file identifies all Services that you have written to execute on a particular TSF instance. There are 2 types of Services that the TSF supports - a persistent service that we just refer to as a "Service", and a "Dynamic Service". These parameters do not have a default value -- one of these values (or both) must be overriden when a TSF instance starts up.

These behave as follows:

Service: Our persistent service capability enables you to write a Service -- a class that you create -- to execute on one or more TSF Servers. The TSF pre-instantiates an instance of your class for each connection that the TSF server supports. (Every TSF Instance [i.e., server] is configured to support a maximum number of connections particular to that server.) During startup, the TSF will create an instance of your class (Service) for each connection. These Services enable stateful interaction (where a command may depend on results of other commands sent prior) and also don't incur extra startup costs every time a new connection arrives.

A typical design pattern is to have your Service create/use a singleton to enable it to share data across connections while using the class you will specify in this configuration to store state particular to the single connection that has connected to your Service through the TSF.

Dynamic Service: These work more like micro-services. A class you write that is a Dynamic Service will be started by a TSF Instance when a command comes in from a connected client that calls for it. That command will cause an instance of your Service to be instantiated; that instance of the Service so instantiated will be used to execute the command and return a response; and, that instance of your Service will then be discarded.

Below is a snippet of the TSF configuration file for a larger grid example to describe the capabilities more fully. Let's suppose that we had a TSF grid of 60 machines, and every machine was running the MktDataService, but only machines 12, 13, 24, and 25 were running the EchoService, and only machines 21, 22, 23, 24, and 25 were running the RiskService (in addition to the MktDataService). Each machine is also configured to run the DynamicEchoService. Since each TSF instance in the grid provides information about the services that it is running, their services and dynamicServices sections would look like the following:

Instances 24, 25

"services": [
   {
     "name": "EchoService", 
     "executorClass": "com.tessellation.tsfservices.EchoService", 
     "runAsSeparateServiceOnEachInstance": false
   },
   {
     "name": "RiskService", 
     "executorClass": "com.bankOfEuripides.capitalMarkets.RiskService", 
     "runAsSeparateServiceOnEachInstance": false,
     "configFile": "/configFilePath/RiskService.cfg path"
   },
   {
     "name": "MktDataService", 
     "executorClass": "com.bankOfEuripides.capitalMarkets.MktDataService", 
     "runAsSeparateServiceOnEachInstance": false,
     "configFile": "/home/tsfuser/app/conf/global/tsCacheMktData.ccfg"
    }
 ],
 "dynamicServices": [
   {
     "name": "DynamicEchoService", 
     "executorClass": "com.tessellation.tsfservices.DynamicEchoService"
   }
 ],

Instances 12, 13

"services": [
   {
     "name": "EchoService", 
     "executorClass": "com.tessellation.tsfservices.EchoService", 
     "runAsSeparateServiceOnEachInstance": false
   },
   {
     "name": "MktDataService", 
     "executorClass": "com.bankOfEuripides.capitalMarkets.MktDataService", 
     "runAsSeparateServiceOnEachInstance": false,
     "configFile": "/home/tsfuser/app/conf/global/tsCacheMktData.ccfg"
    }
 ],
 "dynamicServices": [
   {
     "name": "DynamicEchoService", 
     "executorClass": "com.tessellation.tsfservices.DynamicEchoService"
   }
 ],

Instances 21, 22, 23

"services": [
   {
     "name": "RiskService", 
     "executorClass": "com.bankOfEuripides.capitalMarkets.RiskService", 
     "runAsSeparateServiceOnEachInstance": false,
     "configFile": "/configFilePath/RiskService.cfg path"
   },
   {
     "name": "MktDataService", 
     "executorClass": "com.bankOfEuripides.capitalMarkets.MktDataService", 
     "runAsSeparateServiceOnEachInstance": false,
     "configFile": "/home/tsfuser/app/conf/global/tsCacheMktData.ccfg"
    }
 ],
 "dynamicServices": [
   {
     "name": "DynamicEchoService", 
     "executorClass": "com.tessellation.tsfservices.DynamicEchoService"
   }
 ],

Rest of the instances

"services": [
   {
     "name": "MktDataService", 
     "executorClass": "com.bankOfEuripides.capitalMarkets.MktDataService", 
     "runAsSeparateServiceOnEachInstance": false,
     "configFile": "/home/tsfuser/app/conf/global/tsCacheMktData.ccfg"
    }
 ],
 "dynamicServices": [
   {
     "name": "DynamicEchoService", 
     "executorClass": "com.tessellation.tsfservices.DynamicEchoService"
   }
 ],

When each of these configurations from each TSF instance on start up are processed, it is known that the overall TSF grid is configured to be executing 3 Services and 1 Dynamic Service, and it will be known to all TSF instances in the grid which services are running on which machines in case it gets a request for a service that is not executing on that specific TSF instance, but on a different one in the grid.

Note the "runsAsSeparateServiceOnEachInstance" value of 'false' in the above snippets. If it were 'true', e.g., for the "EchoService", then: Rather than a Service known to the TSF as "EchoService" execute on the 4 TSF Instances designated here, instead: "EchoService_1" would execute on TSF:12, "EchoService_2" would execute on TSF:13, "EchoService_3" on TSF:24 and "EchoService_4" on TSF:25. There are reasons why one might want this value to be false as it is shown: In that case, a request coming in to the TSF from any client connected to any TSF Instance in the grid would be routed to the least expensive TSF Instance to reach. E.g., if client code happened to connect to TSF:10 and sent a command to the "EchoService", because "EchoService" was not executing on TSF:10, the TSF would determine which of TSF:12, TSF:13, TSF:24 or "TSF:25" to send the request to (where said choice would be made using the NetworkModel routing to the least expensive TSF Instance [or randomly among the least expensive to reach TSF Instances]). If true, the connected client code would have to designate to send the request to "EchoService_1", or "EchoService_2", etc.. One reason to use true is when you have designed your Service such that data is sharded across your Service Instances or if your application logic wants to send particular data to particular instances. You can, of course, use the same "executorClass" with different names. For example, if you wanted to use the same TSCache code but have one cache for MktData and one for Trades, you could define two Services as follows (w/MktDataCache executing on all 60 instances and TradeCaches executing on only 10 instances -- 1, 2, 3, 4, 5, 21, 22, 23, 24, 25):

Instances 1, 2, 3, 4, 5, 21, 22, 23, 24, 25

"services": [
   {
     "name": "MktDataCache", 
     "executorClass": "com.tessellation.tsCache", 
     "runAsSeparateServiceOnEachInstance": false,
     "configFile": "/home/tsfuser/app/conf/global/tsCacheMktData.ccfg"
    },
   {
     "name": "TradeCache", 
     "executorClass": "com.tessellation.tsCache", 
     "runAsSeparateServiceOnEachInstance": false,
     "configFile": "/home/tsfuser/app/conf/global/tsCache1.ccfg"
   }
 ],

Rest of the instances

"services": [
   {
     "name": "MktDataCache", 
     "executorClass": "com.tessellation.tsCache", 
     "runAsSeparateServiceOnEachInstance": false,
     "configFile": "/home/tsfuser/app/conf/global/tsCacheMktData.ccfg"
    }
 ],

To see how you would write your Services, please see Server Services section of this documentation.

Configuring Setup ​

Recall the command you ran to start the single TSF server instance in the example.

docker run -v /Users/$USER/Tessellation/POC/setup/overrides/tsf-server:/home/tsfuser/tsf/conf/overrides \
-d --network test-tsf --ip 10.0.0.2 --name tsf-instance-1 -e INSTANCE_NUM=1 \
-e SINGLETON=<isSingletonGrid> -e OVERRIDE_INSTANCE_CONFIG=/home/tsfuser/tsf/conf/overrides/overrideInstanceConfig.json \
-p 9081:9080 tsf/tsf-server-example:test

Add TSF Servers to Grid ​

As mentioned, you can add more TSF Servers to your grid. If you are not using Docker, you cannot start more instances on the same physical machine, since only 1 TSF server can run on a single JVM. If you are using Docker AND you ran the grid using a Register Service (e.g. setting the SINGLETON flag to false), you can add more servers. To do so, run the exact same command you ran in the example, just with the following adjustments:

• Change the ip address from 10.0.0.2 to 10.0.0.3
• Change the name from tsf-instance-1 to tsf-instance-2
• Change the environment variable INSTANCE_NUM from 1 to 2
• Change the port line from -p 9081:9080 to -p 9082:9080

After running this, you should see that it adds itself to the configuration, and after 60 seconds (this time is configurable), you should see the UP_AND_RUNNING message from your new instance. You can now try and send requests to either instance in the grid, and watch as they re-direct some requests to the other if that server does not have that particular service running on it.

JMX ​

We provide the ability to connect to any of your TSF instances in your grid, using JMX properties. Due to its overhead, it is by default not enabled. To enable it, you can tweak your run command as follows:

docker run -v /Users/$USER/Tessellation/POC/setup/overrides/tsf-server:/home/tsfuser/tsf/conf/overrides \
-d --network test-tsf --ip 10.0.0.2 --name tsf-instance-1 -e ENABLE_JMX=true \
-e JMX_HOSTNAME=localhost -e INSTANCE_NUM=1 -e SINGLETON=true \
-e OVERRIDE_INSTANCE_CONFIG=/home/tsfuser/tsf/conf/overrides/overrideInstanceConfig.json \
-p 9081:9080 tsf/tsf-server-example:test

Now, through any terminal, you can type 'jconsole' and a gui pops up, where you can connect to your TSF instance at: <hostname>:<7070 + TSF instance number>, so in the above example, it would be localhost:7071. This will allow you to see all of the currently active threads running in the container.

If you want to change the base port that the port is calculated from (by default 7070), you can change it in the properties file.

E.g.

JMX_BASE_PORT=8080

in which case, the connection in the example above would be localhost:8081

Secure Connection ​

This property allows connections to a TSF instance to be secure - by default it is false. More information on what this means and how it works is found in the Client documentation.

docker run -v /Users/$USER/Tessellation/POC/setup/overrides/tsf-server:/home/tsfuser/tsf/conf/overrides \
-d --network test-tsf --ip 10.0.0.2 --name tsf-instance-1 -e INSTANCE_NUM=1 \
-e SINGLETON=true -e SECURE=true -e OVERRIDE_INSTANCE_CONFIG=/home/tsfuser/tsf/conf/overrides/overrideInstanceConfig.json \
-p 9081:9080 tsf/tsf-server-example:test

Note: If you are running the TSF without docker, all of the information above is the same, except instead of modifying the environment variables as we are doing above, you need to modify the corresponding fields in the start.sh script that you ran (See the start.sh script in the Register Service non docker tar for a complete list of all configurable properties).

To see the full list and description of all of the possible configurable server parameters, click here.