7. External Connector Configuration

The main configuration file of the external connector is the ext_conn_config.ini file, but the name can be changed if the configuration file is passed as command line option (as explained in External Connector Installation & Execution). Just keep in mind that the file should be always a ini file. In this file you can configure those aspects that are relevant for only the external connector such as how to connect with the engine, the use of certificates for TLS support, the logs, different times, the connection with the Redis server (its role is explained in Architecture), and other operation configuration.

7.1. Engine Connection

The first thing you have to configure is where to find the engine:

# ==========================================================================
[NETWORKING]
# ==========================================================================
# ZEROMQ
# IP or domain
ENGINE_IP = 127.0.0.1
ENGINE_PORT = 5555

7.2. External Connector Logging

The next interesting section to configure is the logs of the external connector. In the same way that the engine can generate logs, there are several possibilities:

• Console logging: The logs are only showed via the terminal.

• File logging: The logs are saved in a single file.

• Console & file logging: Logs are saved in a file, as well as shown via console.

• File logging with log rotation: In this case, the logs are rotated in several files depending on the configuration parameters.

• Console & file with log rotation: Logs are saved in several files as described in the previous point, as well as shown via console.

The created log files have the following name structure: slv_ext_conn_%Y_%m_%d-%H_%M_%S.log (example: slv_ext_conn_2022_09_21-08_29_11.log). But the prefix and file extension can be changed in the configuration file. If the specified log folder does not exist, it will be created.

# ==========================================================================
[LOGGING]
# ==========================================================================
# LEVEL: one of the following: DEBUG, INFO (default), WARNING, ERROR, CRITICAL
LEVEL = DEBUG
# LOG_Folder: not include the final '/', for local folder use '.'
# It creates the the folder if it does not exist
LOG_FOLDER = ./logs
# LOG_MODE: CONSOLE (default), FILE, FILE_ROTATION, BOTH, BOTH_ROTATION
LOG_MODE =  CONSOLE
# Log rotation parameters, by default the max size is 20 MB and 5 files
LOG_ROTATION_MAX_SIZE = 20
LOG_ROTATION_FILE_NUMBERS = 5
# Log file name utils. if not used, LOG_PREFIX = "slv_ext_conn_" and LOG_EXTENSION = ".log"
LOG_PREFIX =
LOG_EXTENSION =

7.3. External Connector Timing

For the external connector, timing is an important aspect. Depending on the context, the different values should be adapted.

# ==========================================================================
[TIMES]
# ==========================================================================
# Timeout for the connection with the engine
# 5s by default
ENGINE_TIME_OUT = 5

# Timeout (seconds) for the external connector (lifespan),
# 'NONE' for not using a timeout
# by default = 1 hour = 3600 seconds
EXT_CONNECTOR_TIME_OUT = NONE

# Time (seconds) for main thread operation loop without contacting
# the engine in case if there is not
# information to send to the engine. By default = 5 seconds.
# If this value is less than the
# ENGINE_TIME_OUT, then this last value is used
# for the loop waiting time
MAIN_THREAD_CHECKING_TIME = 5

# Time (float-seconds) between loops when the external connector
# wait for interaction The default value is 0.5 seconds
TIME_BETWEEN_INTERACTION_LOOPS = 0.5

# Time (float seconds) between loops for the async thread.
# The default value us 0.1 seconds
TIME_BETWEEN_ASYNC_LOOPS = 0.10

# Number of async loops to clean the connection register.
# It results in a time value depending on
# the time between async loops
NUMBER_ASYNC_LOOPS_TO_CLEAN_CONN_REGISTER = 10000

# Waiting time (int-seconds) before restarting
# the external connector according to 'restart' order
RESTARTING_WAITING_TIME = 60

# Time (float-seconds) between connections of
# client sockets during starting-up (by deafult = 0.5s)
TIME_BETWEEN_CLIENT_SOCKET_CONNECTIONS = 0.5

# Time (float-seconds) between closing a tcp connection
# and connecting again for tcp client mode (by deafult = 0.5s)
# Or initial delay for having enough time to do the
# TLS handhsake in the reconnection
TIME_BETWEEN_CLIENT_SOCKET_CLOSE_CONNECT = 1

# Time (float-seconds) before allowing the async tasks in
# client mode checks if no active connections are alive (by deafult = 3s)
# Recommended: 5s or more for several TLS or DTLS connections.
TIME_INITIAL_DELAY_ASYNC_THREAD = 5

7.4. External Connector Operation

The following parameters cover other operational aspects of the external connector. The first two are the credentials to authenticate the external connector against the engine, but if other values will replace them if they are provided via command line options, as described in External Connector Installation & Execution. The third one is for being used to authenticate the engine against the external connector, as mentioned in Engine Installation & Execution.

The last two parameters allow to find the custom code that you can add to customize even further the interaction. This is described in detail in Custom Functions.

# ==========================================================================
[OPERATION]
# ==========================================================================

# external connector ID, by default is autogenerated (AUTO, or empty)
EXTERNAL_CONNECTOR_ID = ExtConTest

# API key for connecting with the engine
EXTERNAL_CONNECTOR_SECRET = 3sT0_3S_s3Cr3t0

# For the hash code sent to external connectors from
# the engine as authentication proof
ENGINE_AUTH_CODE = Y0_s0y_0zym4nd14s_r3y_d3_r3y3s

# Custom functions directory. The software is going to
# look for a python module called with the given name
# ('slv_custom_functions[.py]' in this case). If it is not found,
# no custom functions will be executed. Please do not
# add the extension .py in the name
CUSTOM_FUNCTIONS_DIRECTORY = ../custom_functions/
CUSTOM_FUNCTIONS_MODULE_NAME = slv_custom_functions

# Number of subworkers to divide the task of checking all the regex of custom
# rules for one input. This only is used when the external connector does not
# work in 'only first hit' scenario
NUMBER_RULE_CHECKER_SUBWORKERS = 10

7.5. TLS/DTLS Support

In case the external connector needs to use TLS for tcp, or DTLS for udp, this will be the place for configuring it. The support of TLS includes the client authentication, as well as the case the private key is encrypted; but always using material created as OpenSSL does.

It is important to mention that for the DTLS case and if the private key is protected, a temporary file will created with the unencrypted key for the exection and deleted once the execution ends.

The following links can help you to configure the TLS or DTLS correctly for the external connector:

• TLS support in Python offical documentation.

• OpenSSL tutorial from Jamie Nguyen.

• Certificate chains in Python offical documentation.

• OpenSSL cheat sheet from Cédric Lauradoux.

• Problems using TLS in Python (ssl.SSLCertVerificationError).

• Working with OpenSSL and DNS alternative names.

• Key usage extensions and extended key usage.

• How to create an HTTPS certificate for localhost domains.

# ==========================================================================
[D/TLS]
# ==========================================================================
# https://docs.python.org/3/library/ssl.html#module-ssl

# Custom CA:
# - in server mode, it is the CA for validating client certificates
# - in client mode, it is the CA for validating the server certificate if default
#   settings are not used
CLIENT_CABUNDLE_PEM = ../tls/ca/certs/ca.crt.pem

# Is the key of the certificate to use protected with a password? YES (default) / NO
KEY_PROTECTED = YES

# ---------------------
# TLS SERVER MODE
# ---------------------
# https://docs.python.org/3/library/ssl.html#server-side-operation
SERVER_CERTCHAIN_PEM = ../tls/ca/intermediate/certs/ca-chain_server.crt.pem
SERVER_PRIV_KEY_PEM = ../tls/ca/intermediate/private/seclopedevega_server.key.pem
SERVER_KEY_PASSWORD = lope

# ---------------------
# TLS CLIENT MODE
# ---------------------
# https://docs.python.org/3/library/ssl.html#client-side-operation
# Use Machine CAs: YES (default) / NO
CLIENT_DEFAULT_SETTINGS = NO
# Validate Server certificate: YES (default) / NO
CLIENT_VALIDATE_SERVER_CERTIFICATE = YES
CLIENT_CERTCHAIN_PEM = ../tls/ca/intermediate/certs/ca-chain_client.crt.pem
CLIENT_PRIV_KEY_PEM = ../tls/ca/intermediate/private/seclopedevega_client.key.pem
CLIENT_KEY_PASSWORD = lope

# ---------------------
# DTLS MODE
# ---------------------
#https://github.com/mcfreis/pydtls
# One of the following: "DTLS" or "DTLSv1.2" (default)
DTLS_VERSION = DTLSv1.2

7.6. Redis Integration

As described in Architecture, the Redis server is an important element for allowing to share information among different external connectors, using memory variables. These variables are explained in detail in Session Support & Memory Variables, here we only address what is the configuration capabilities to use Redis.

he following links can help you to install and use Redis, as well as understaning the integration between the external connector and Redis:

• How to Use Redis With Python.

• How To Install and Secure Redis on Ubuntu 20.04.

• Redis TLS support.

• Redis-py TLS Connection Examples.

# ==========================================================================
[REDIS] # Multi external connector memory
# ==========================================================================
# IP or domain
REDIS_IP = 127.0.0.1

REDIS_PORT = 6379

REDIS_PASSWORD = 3st33s3lB4uLD3L4sV4r14Bl3sC0mP4Rt1D4s

REDIS_TLS = NO

# use certificate for authenticate against the redis server?
REDIS_USE_CLIENT_CERTIFICATE = YES
REDIS_CLIENT_CERTIFICATE = ../tls/ca/intermediate/certs/ca-chain_client.crt.pem

# Private Key fields for the certificate
REDIS_PRIV_KEY_CLIENT_PROTECTED = YES
REDIS_PRIV_KEY_PASSWORD =  lope
REDIS_PRIV_KEY_CLIENT = ../tls/ca/intermediate/private/seclopedevega_client.key.pem

# Machine CAs in linux: /etc/ssl/certs/ca-certificates.crt
REDIS_CA_CERT = ../tls/ca/certs/ca.crt.pem