Component diagram of a system for the automation of experimental or analytical setups.

Figure 28. Component diagram of the System for Experiments Control.

At the highest level we identified the need of a controller module (Experiment Controller) responsible for controlling the start, the progress and the end of a experiment, or a analytical procedure.

This control is based on a script of several tasks, defined by the user, and performed according to the status and/or elapsed time.

The Experiment Controller is a client which depends on 3 servers: Monitoring Server, Database Server and Controlling Server.

Monitoring Server: Centralizes the flow of data from various “instruments” to the Experiment Controller.

The Database Server: Centralizes the storage and access of information about an experiment.

Controlling Server:Enables the integrated use of many “equipments” during an experiment.

The relationship between the servers (monitoring, database and controlling) and the experiment controller are modeled according to the standard LECIS.

The user should be permitted, within certain limits, to change the experimental script during the experiment.

This configuration allows exeperimental setups with only monitoring activities, or just control or both.

For example, in respirometric measurements of biological sludge (from aerobic sewage treatment) we need to measure the comsuption of Dissolved Oxygen (DO) controlling the aeration and nutrient dosing.

In this case the Experiment Controller receives the measures of DO, pH and temperature from Monitoring Server. Then stores these measurements in the database by Database Server and sends commands to the Controlling Server to trigger the aeration pump, dosing pump and, eventually, a heater.

Such control of the experiment can be time-dependent, system-dependent or both.

Figure 29. Component diagram with additional modules connected to the Experiment Controller by TCP sockets.

The pH control may be done by Experiment Controller following rules from a “Module for Analysis and Data Interpretation” or be “transferred” to the Controlling Server following a simple ON/OFF control.

The Experiment Controller must be able to perform a list of pre-defined tasks, but these tasks can be changed at any time. This configuration aims to provide flexibility to control experiments for monitoring and/or control.

That is, the “Monitoring Server” controls “instruments” and the “Control Server” controls “equipments” (which do not provide information about the system under study).

The monitoring and controlling servers can also have a dedicated database to store parameter for configuration.

Figure 30. Component diagram of Monitoring Server with many “Client Instruments” and the client “Experiment Controller”.

Each instrument is controlled by a client program (Standard Laboratory Module - SLM) that communicates with the Monitoring Server, which centralizes the communication between instruments (SLM) and Experiment Controller (Task Sequence Control - TSC).

Each instrument client has a configuration file (Device Capability Dataset - DCD) with information about the Monitoring Server (host and port), arguments of each standard command and event report, physical characteristics of instrument, resources that it uses in its tasks and definition of optional interactions.

The Monitoring Server uses a port (A) to communicate with the controller and another port number to connect with instruments. Thus using TCP sockets we may have Instruments, Monitoring Server and Experiment Controller running on different machines on a network.

Considering the standard recommendations of LECIS we can model the initial stages of an experiment according to the sequence diagram 31

Figure 31. Sequence diagram of the initial stages of an experiment.

And in figure 32 a more specific sequence diagram for configuration of instruments.

Figure 32. Sequence diagram for configuration of instruments during the initial stages of an experiment.

To organize the files on the system was created the directory programaro and subdirectories:

And in programaro the files:

We began to code the server instkont_00.tcl with the following initial requirements:

instkont_00.tcl:

#!/usr/bin/env tclsh


#This program (InstKont - Instrument Controller)
# was developed to work as a server to centralize 
#the data acquisition of many analytical instruments commonly used 
#in monitoring of water quality.

#Este programa (InstKont - Controlador de Instrumentos) foi desenvolvido 
#para operar como um servidor centralizando a aquisição de dados 
#de vários intrumentos analíticos comumente usados no 
#monitoramento da qualidade da água


#    "Copyright (C) 2016  Markos (markos[at]c2o.pro.br)"

#    This program is free software: you can redistribute it and/or modify
#    it under the terms of the GNU General Public License as published by
#    the Free Software Foundation, either version 3 of the License, or
#    (at your option) any later version.

#    This program is distributed in the hope that it will be useful,
#    but WITHOUT ANY WARRANTY; without even the implied warranty of
#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#    GNU General Public License for more details.

#    You should have received a copy of the GNU General Public License
#    along with this program.  If not, see http://www.gnu.org/licenses/.

#Ref. Tcl 8.5 Network Programming / Wojciech Kocjan and Piotr Beltowski
#page 136


#Types of log events 
#(Ref: http://docs.fortinet.com/fmgr/fortimanager-lmr-40-mr3.pdf)
#Level Description
#0 emergency - The system has become unusable.
#1 alert - Immediate action is required.
#2 critical - Functionality is affected.
#3 error - An erroneous condition exists, and functionality is probably affected.
#4 warning - Functionality could be affected.
#5 notice - Information about normal events.
#6 info - General information about system operation.
#7 debug - Detailed information useful for debugging purposes.

#Package to help logging and debugging
package require log

#Replacing the default implementation of the logging command

#Enabling all log levels
#Ativando todos os níveis de log
log::lvSuppressLE warning 0

#Replacing default log::Puts by the command (myPuts) 
#Substituindo o comando padrão log::Puts pelo comando myPuts
log::lvCmdForall myPuts

proc myPuts {lvl msg} {
    set now [clock seconds]
    set date [clock format $now]
    set channel [log::lv2channel $lvl]
    puts $channel [format "\[%s\] \[%s\] %s" \
        $date $lvl $msg]
    flush $channel
}

#TclOO is an object system for Tcl
#http://sourceforge.net/projects/tcl/files%2FTclOO%20Package/
package require TclOO



#Creating safe interpreters to execute only the commands defined for each interpreter
#Criando um interpretador seguro para executar apenas os comandos definidos para cada
#interpretador

#Ref. Effective Tcl/Tk Programming: Writing Better Programs with Tcl and Tk
#page 286 (Safer Parsing)


#Safe parser for Instruments
set parserInst [interp create -safe]

$parserInst eval {

    proc add {x y} {
	return [expr $x + $y]
    }

}

#Safe parser for the client "Experiment Controller"
set parserEkspKont [interp create -safe]

$parserEkspKont eval {

#availableInst return a list of all files with "extension" 
#in a directory "instrumentoj"

    proc availableInst {} {
	
	set list_inst [ cmdListFiles instrumentoj tcl ]
	
	if { $list_inst == "" } {
	    set list_inst "No instrument available"
	    return $list_inst
	} else {
	    return $list_inst
	}

    }

 
}

#Defines a procedure called listFiles in the main interpreter and 
#then adds an alias called cmdListFiles in the safe interpreter.
#The command cmdListFiles is called in the safe interpreter and
#the command then transfer control to the real listFiles in the main
#interpreter where the command "glob" can be executed.

#Define um procedimento chamado listFiles no interpretador principal
#e adiciona um "alias" (apelido, link) chamado cmdListFiles no interpretador
#seguro. O comando cmd_listFiles é chamado no interpretador seguro e
#o comando então transfere o controle para o listFiles real no 
#interpretador principal onde o comando "glob" pode ser executado.

#Generic procedure to return all files with "extension" 
#in a directory "dir"

#Procedimento genérico para retornar todos os arquivos com a
#extensão "extension" presentes em um diretório "dir"

proc listFiles { dir extension } {

set d "*."

set extension [append d $extension]

set list_files [glob -nocomplain -directory $dir $extension]

return $list_files

}

$parserEkspKont alias cmdListFiles listFiles


#dict variable with "addr", "port" and "service" keys
set clientInst [dict create]

set clientEkspKont [dict create]

#Ref. Tcl/Tk : a developer's guide / Clif Flynt. 3rd ed.
#page 110

#-----------------------------------------------------------------------------
#Defining the port 1901 for the experiment controller (kontrolanto de la eksperimento)
#and 1902 for the instruments
#Definindo a porta 1901 para o controlador do experimento (kontrolanto de la eksperimento)
#e a porta 1902 para os instrumentos

set portEkspKont 1901
set portInst 1902

proc serverInstOpen {server channel client port} { 
    
    global clientInst

# Set up fileevent to be called when input is available
# Define o procedimento que será executado quando houver dados no canal

    fileevent $channel readable [list readInst $channel] 
    chan configure $channel -buffering line -blocking 0
    
    dict set clientInst $channel addr $client
    dict set clientInst $channel port $port
    dict set clientInst $channel service $server
    
    log::log info "Connection from ${client}:${port} to $channel (service=$server)"
    
} 

#Procedure to be called when input is available from an Instrument
#Procedimento que será executado quando houver dados no canal de um Instrumento

proc readInst {channel} { 

    global parserInst clientInst
    
    set len [gets $channel request] 
    # if we read 0 chars, check for EOF. Close the 
    # channel and delete that client entry if we've 
    # hit the end of the road. 
    if {($len <= 0) && [eof $channel]} { 
	set client "[dict get $clientInst $channel addr]:[dict get $clientInst $channel port]"
	log::log warning \
	    "Instrument from $client\
             Disconnected from channel $channel"
	dict unset clientInst $channel
	close $channel
	log::log debug "Clients connected: [dict get $clientInst]"
    } else { 
	# Process request
	if {[catch {$parserInst eval $request} result] == 0} {
	    chan puts $channel $result
	    log::log info \
	    "Request:$request from \
    [dict get $clientInst $channel addr]:[dict get $clientInst $channel port] \
    to channel:$channel resulted:$result"
	} else {
	    chan puts $channel [list error_result $result]
	    log::log error "Request:$request from \
    [dict get $clientInst $channel addr]:[dict get $clientInst $channel port] \
    to channel:$channel resulted:$result"
	} 
    }
}


proc serverEkspKontOpen {server channel client port} { 
    
    global  clientEkspKont 
    
    # Set up fileevent to be called when input is available
    # Define o procedimento que será executado quando houver dados no canal
    
    fileevent $channel readable [list readEkspKont $channel] 
    chan configure $channel -buffering line -blocking 0
    
    dict set clientEkspKont $channel addr $client
    dict set clientEkspKont $channel port $port
    dict set clientEkspKont $channel service $server
    
    log::log info "Connection from ${client}:${port} to $channel (service=$server)"
    
    
} 

#Procedure to be called when input is available from Experiment Controller
#Procedimento que será executado quando houver dados no canal do Controlador de Experimento

proc readEkspKont {channel} { 
    
    global parserEkspKont clientEkspKont
    
    set len [gets $channel request] 
    # if we read 0 chars, check for EOF. Close the 
    # channel and delete that client entry if we've 
    # hit the end of the road. 
    if {($len <= 0) && [eof $channel]} { 
	set controller "[dict get $clientEkspKont $channel addr]:[dict get $clientEkspKont $channel port]"
	log::log warning \
	    "Experiment controller from $controller\
             Disconnected from channel $channel"
	dict unset clientEkspKont $channel
	close $channel 
    } else { 
	# Process request
	if {[catch {$parserEkspKont eval $request} result] == 0} {
	    chan puts $channel $result
	    log::log info \
	    "Request:$request from \
    [dict get $clientEkspKont $channel addr]:[dict get $clientEkspKont $channel port] \
    to channel:$channel resulted:$result"
	      
	} else {
	    chan puts $channel [list error_result $result]
	    log::log error "Request:$request from \
    [dict get $clientEkspKont $channel addr]:[dict get $clientEkspKont $channel port] \
    to channel:$channel resulted:$result"	  
	} 
    }
}

set serverInst [socket -server {serverInstOpen serverInst} $portInst]

set serverEkspKont [socket -server {serverEkspKontOpen serverEkspKont} $portEkspKont]

proc getDCD {} {
    
    return "return getDCD"
    
}

proc configInst {args} {
    
    return "return configInst"
    
}



# Initialize a variable and wait for it to change to keep tclsh running
# Inicializa uma variável e espera a sua modificação para manter o tclsh rodando

set running 1
vwait running

In a terminal the server is started with the command ./instkont_00.tcl

And to make the first tests we used the Telnet program making connections to ports 1901 and 1902:

  $ telnet localhost 1901
Trying ::1...
Connected to localhost.
Escape character is '^]'.

And to port 1902:

  $ telnet localhost 1902
Trying ::1...
Connected to localhost.
Escape character is '^]'.

And the log of server:

$ ./instkont_00.tcl
[Wed Mar 30 11:26:59 UTC 2016] [info] Connection from ::1:33061 to sock8220848 (service=serverEkspKont)
[Wed Mar 30 11:27:42 UTC 2016] [info] Connection from ::1:48888 to sock82265d8 (service=serverInst)

Was created the procedure add within the safe interpreter parserInst, and the function availableInst inside the the safe interpreter parserEkspKont.

When these functions are called by the clients the corresponding returns are sent.

To port 1901:

  telnet localhost 1901
Trying ::1...
Connected to localhost.
Escape character is '^]'.
availableInst
instrumentoj/instrumento.tcl

To port 1902:

  $ telnet localhost 1902
Trying ::1...
Connected to localhost.
Escape character is '^]'.
add 2 3
5

And the log from server:

  [Wed Mar 30 12:40:09 UTC 2016] [info] Request:availableInst from  ::1:33224  to channel:sock88ec848 resulted:instrumentoj/instrumento.tcl
  [Wed Mar 30 12:46:33 UTC 2016] [info] Request:add 2 3 from  ::1:49061  to channel:sock97c15d8 resulted:5

To check security we left the procedures getDCD and configInst out of safe parsers: parserInst and parserEkspKont.

When these functions are called by the clients the corresponding returns are sent by the server.

To port 1901:

    $ telnet localhost 1901
Trying ::1...
Connected to localhost.
Escape character is '^]'.
getDCD
error_result {invalid command name "getDCD"}
configInst
error_result {invalid command name "configInst"}

To port 1902:

  $ telnet localhost 1902
Trying ::1...
Connected to localhost.
Escape character is '^]'.
getDCD
error_result {invalid command name "getDCD"}
configInst
error_result {invalid command name "configInst"}

And the log from server:

[Wed Mar 30 12:52:35 UTC 2016] [error] Request:getDCD from  ::1:33241  to channel:sock9955848 resulted:invalid command name "getDCD"
[Wed Mar 30 12:52:39 UTC 2016] [error] Request:getDCD from  ::1:49068  to channel:sock995b5d8 resulted:invalid command name "getDCD"
[Wed Mar 30 12:55:21 UTC 2016] [error] Request:configInst from  ::1:33241  to channel:sock9955848 resulted:invalid command name "configInst"
[Wed Mar 30 12:55:30 UTC 2016] [error] Request:configInst from  ::1:49068  to channel:sock995b5d8 resulted:invalid command name "configInst"

It works. :^)

But the way it is there is the possibility of more than one experiment controller client to connect to the monitoring server.

So it is necessary to establish a mechanism to allow the connection of “only one experiment controller client” at a time.

Following the suggestions of comp.lang.tcl and the tutorial Simple TCP/IP to serial port gateway I decided to use the dict variable client_Eksp_Kont as a “semaphore” to control the connection of only one experiment controller client at a time.

To do this I included an “if” condition in the function serverEkspKontOpen:

  proc serverEkspKontOpen {server channel client port} { 
    
    global  client_Eksp_Kont 
    
    #Close the channel if another client has connected
    #Fecha o canal se já houver um cliente conectado
    
    if {$client_Eksp_Kont ne ""} {
	log::log warning "Server $server refusing connection from client $client, channel $channel and port $port."
	chan puts $channel "Sorry, already have a client"
	close $channel
	return
	}

	...

And deleting the contents of the variable client_Eksp_Kont at the end of the connection in procedure readEkspKont.

  if {($len <= 0) && [eof $channel]} { 
	set controller "[dict get $client_Eksp_Kont $channel addr]:[dict get $client_Eksp_Kont $channel port]"
	log::log warning \
	    "Experiment controller from $controller disconnected from channel $channel"
	log::log info "Closing connection with client_Eksp_Kont: $client_Eksp_Kont"
	dict unset client_Eksp_Kont $channel
	close $channel