The Sipwise C5 provides a configuration framework for consistent and easy to use low level settings management. A basic usage of the configuration framework only needs two actions already used in previous chapters:
Low level management of the configuration framework might be required by advanced users though. This chapter explains the architecture and usage of Sipwise C5 configuration framework. If the basic usage explained above fits your needs, feel free to skip this chapter and return to it when your requirements change.
A more detailed workflow of the configuration framework for creating a configuration file consists of 7 steps:
The Sipwise C5 provides configuration file templates for most of the services it runs. These templates are stored in the directory /etc/ngcp-config/templates.
Example: Template files for /etc/ngcp-sems/sems.conf are stored in /etc/ngcp-config/templates/etc/ngcp-sems/.
There are different types of files in this template framework, which are described below.
These files are the main template files that will be used to generate the final configuration file for the running service. They contain all the configuration options needed for a running Sipwise C5 system. The configuration framework will combine these files with the values provided by config.yml, constants.yml and network.yml to generate the appropriate configuration file.
Example: Let’s say we are changing the IP used by kamailio load balancer on interface eth0 to IP 1.2.3.4. This will change kamailio’s listen IP address, when the configuration file is generated. A quick look to the template file under /etc/ngcp-config/templates/etc/kamailio/lb/kamailio.cfg.tt2 will show a line like this:
listen=udp:[% ip %]:[% kamailio.lb.port %]
After applying the changes with the ngcpcfg apply 'my commit message' command, a new configuration file will be created under /etc/kamailio/lb/kamailio.cfg with the proper values taken from the main configuration files (in this case network.yml):
listen=udp:1.2.3.4:5060
All the low-level configuration is provided by these .tt2 template files and the corresponding config.yml file. Anyway, advanced users might require a more particular configuration.
Instead of editing .tt2 files, the configuration framework recognises .customtt.tt2 files. These files are the same as .tt2, but they have higher priority when the configuration framework creates the final configuration files. If you need to introduce changes in a template, you must always copy the required .tt2 file to .customtt.tt2, make changes in the latter file one and leave the .tt2 file untouched. This way, the system will use the new custom configuration allowing you to switch back to the original one quickly.
Example: We’ll create /etc/ngcp-config/templates/etc/lb/kamailio.cfg.customtt.tt2 and use it for our customized configuration. In this example, we’ll just append a comment at the end of the template.
cd /etc/ngcp-config/templates/etc/kamailio/lb cp kamailio.cfg.tt2 kamailio.cfg.customtt.tt2 echo '# This is my last line comment' >> kamailio.cfg.customtt.tt2 ngcpcfg apply 'my commit message'
The ngcpcfg command will generate /etc/kamailio/lb/kamailio.cfg from our custom template instead of the general one:
tail -1 /etc/kamailio/lb/kamailio.cfg # This is my last line comment
| warning | |
users have to upgrade all .customtt.tt2 manually every time .tt2 is upgraded, as ngcpcfg completely ignores new code in .tt2 received from new package version. |
The huge drawback of .customtt.tt2 files are necessity to keep them up-to-date manually. Keeping them outdated will cause the system misbehaviour as different components will use different code version (as new .tt2 version will be overwritten by old .customtt.tt2).
The .patchtt.tt2 concept should help users here. It will minimise the manual efforts by using linux "patch" utility. The ngcpcfg tool is searching for .patchtt.tt2 files every time ngcpcfg build has been called. If .patchtt.tt2 is detected, the ngcpcfg tool will try to apply .patchtt.tt2 on .tt2 and store result in .customtt.tt2 if no conflicts noticed during patching. Further building process happens in a common way. Example:
root@spce:~# ngcpcfg build /etc/kamailio/lb/kamailio.cfg spce: yml configs were validated successfully spce: configs were checked successfully spce: Validating patch '/etc/ngcp-config/templates/etc/kamailio/lb/kamailio.cfg.patchtt.tt2' spce: Applying patch '/etc/ngcp-config/templates/etc/kamailio/lb/kamailio.cfg.patchtt.tt2' spce: Successfully created '/etc/ngcp-config/templates/etc/kamailio/lb/kamailio.cfg.customtt.tt2' spce: Requested patchtt operation has finished successfully. Loading /etc/ngcp-config/config.yml in memory: OK Loading /etc/ngcp-config/network.yml in memory: OK Loading /etc/ngcp-config/constants.yml in memory: OK spce: Generating /etc/kamailio/lb/kamailio.cfg: OK spce: Executing postbuild for /etc/kamailio/lb/kamailio.cfg root@spce:~#
To convert some/all the current .customtt.tt2 users can use command ngcpcfg patch --from-customtt [<customtt_file>]:
root@spce:~# ngcpcfg patch --from-customtt /etc/ngcp-config/templates/etc/kamailio/lb/kamailio.cfg.customtt.tt2 spce: Validating customtt '/etc/ngcp-config/templates/etc/kamailio/lb/kamailio.cfg.customtt.tt2' spce: Creating patchtt file '/etc/ngcp-config/templates/etc/kamailio/lb/kamailio.cfg.patchtt.tt2' spce: Requested customtt operation has finished successfully. root@spce:~#
Here is the example of newly created .patchtt.tt2 file:
root@spce:~# cat /etc/ngcp-config/templates/etc/kamailio/lb/kamailio.cfg.patchtt.tt2 @@ -1799,3 +1799,4 @@ } # vim: ft=cfg +# This is my last line comment root@spce:~#
See more details about .patchtt.tt2 files below in patchtt section.
| tip | |
The .tt2 files use the Template Toolkit language. Therefore you can use all the feature this excellent toolkit provides within ngcpcfg’s template files (all the ones with the .tt2 suffix). |
Keeping custom modifications directly in the .customtt.tt2 templates is NOT recommended as templates become outdated with every software upgrade.
A better way is to handle custom modifications using .patchtt.tt2 files (e.g. /etc/ngcp-config/templates/etc/cron.d/cleanup-tools.patchtt.tt2). In this case, on every "ngcpcfg patch", a .patchtt.tt2 file will be applied on top of the .tt2 file and the result will be saved into the customtt file and used commonly as described in the previous section. "ngcpcfg patch" is the first step on "ngcpcfg build" that guarantees the latest upstream templates with the availability of the necessary local changes on every configuration apply.
| tip | |
The patch to be applied to the corresponding .tt2 template file is selected in the following order (highest to lowest): *.patchtt.tt2.$HOSTNAME *.patchtt.tt2.$PAIRNAME *.patchtt.tt2.$HA_NODE *.patchtt.tt2 |
| info | |
If a suitable patchtt file is found for a template, then the ngcpcfg patch command will overwrite the corresponding customtt file, if any. |
Let us see how to introduce custom changes into a template through a patchtt file. For example, we need to change the accounting records cleanup time, which is defined in cleanup-tools.tt2. Here is how to do this:
cd /etc/ngcp-config/templates/etc/cron.d/
cp ./cleanup-tools.tt2 ./cleanup-tools.customtt.tt2
vim ./cleanup-tools.customtt.tt2
ngcpcfg patch --from-customtt ./cleanup-tools.customtt.tt2 cat ./cleanup-tools.patchtt.tt2
ngcpcfg apply "Change acc-cleanup time from 00 to 02 hours" ngcpcfg push all
You will notice that the "ngcpcfg apply" command has generated the customtt file for the corresponding template:
root@web01a:/etc/ngcp-config/templates/etc/cron.d# ls -l ./cleanup-tools* -rw------- 1 root root 932 Jan 4 11:11 ./cleanup-tools.customtt.tt2 -rw-r--r-- 1 root root 630 Jan 4 11:08 ./cleanup-tools.patchtt.tt2 -rw-r--r-- 1 root root 932 Dec 18 15:09 ./cleanup-tools.tt2
Now, even if cleanup-tools.tt2 slightly changes after a software upgrade, "ngcpcfg apply" will still preserve your custom changes.
| info | |
If in a new release the .tt2 file gets changed in the same lines where you had introduced custom changes (e.g. your changes were temporary until a feature is implemented properly in a new software release), the apply process will fail and ask you to review the corresponding .patchtt.tt2 file. Then, check it and either correct if it is still required or remove it. |
| tip | |
To convert all existing customtt files to patchtt files use the command: ngcpcfg patch --from-customtt |
After creating the configuration files, the configuration framework can execute some commands before and after placing that file in its target directory. These commands usually are used for changing the file’s owner, groups, or any other attributes. There are some rules these commands need to match:
Example: We need www-data as owner of the configuration file /etc/ngcp-ossbss/provisioning.conf. The configuration framework will by default create the configuration files with root:root as owner:group and with the same permissions (rwx) as the original template. For this particular example, we will change the owner of the generated file using the .postbuild mechanism.
echo 'chgrp www-data ${output_file}' \
> /etc/ngcp-config/templates/etc/ngcp-ossbss/provisioning.conf.postbuild.services files are pretty similar and might contain commands that will be executed after the build process. There are two types of .services files:
When the services step is triggered all .services files associated to a changed configuration file will be executed. In case of the general file, any change to any of the configuration files in the directory will trigger the execution of the commands.
| tip | |
If the service script has the execute flags set (chmod +x $file) it will be invoked directly. If it doesn’t have execute flags set it will be invoked under bash. Make sure the script is bash compatible if you do not set execute permissions on the service file. |
These commands are usually service reload/restarts to ensure the new configuration has been loaded by running services.
| info | |
The configuration files mentioned in the following example usually already exist on the platform. Please make sure you don’t overwrite any existing files if following this example. |
Example:
echo 'ngcp-service restart mariadb' \ > /etc/ngcpcfg-config/templates/etc/mysql/my.cnf.services echo 'ngcp-service restart asterisk' \ > /etc/ngcpcfg-config/templates/etc/asterisk/ngcpcfg.services
In this example we created two .services files. Now, each time we trigger a change to /etc/mysql.my.cnf or to /etc/asterisk/* we’ll see that MySQL or Asterisk services will be restarted by the ngcpcfg system.
The /etc/ngcp-config/config.yml file contains all the user-configurable options, using the YAML (YAML Ain’t Markup Language) syntax.
The /etc/ngcp-config/constants.yml file provides configuration options for the platform that aren’t supposed to be edited by the user. Do not manually edit this file unless you really know what you’re doing.
The /etc/ngcp-config/network.yml file provides configuration options for all interfaces and IP addresses on those interfaces. You can use the ngcp-network tool for conveniently change settings without having to manually edit this file.
The /etc/ngcp-config/ngcpcfg.cfg file is the main configuration file for ngcpcfg itself. Do not manually edit this file unless you really know what you’re doing.
The ngcpcfg utility supports the following command line options:
The apply option is a short-cut for the options "check && build && services && commit" and also executes etckeeper to record any modified files inside /etc. It is the recommended option to use the ngcpcfg framework unless you want to execute any specific commands as documented below.
The build option generates (and therefore also updates) configuration files based on their configuration (config.yml) and template files (.tt2). Before the configuration file is generated a present .prebuild will be executed, after generation of the configuration file the according .postbuild script (if present) will be executed. If a file or directory is specified as argument the build will generate only the specified configuration file/directory instead of running through all present templates.
Example: to generate only the file /etc/nginx/sites-available/ngcp-panel you can execute:
ngcpcfg build /etc/nginx/sites-available/ngcp-panel
Example: to generate all the files located inside the directory /etc/nginx/ you can execute:
ngcpcfg build /etc/nginx/
The commit option records any changes done to the configuration tree inside /etc/ngcp-config. The commit option should be executed when you’ve modified anything inside the configuration tree.
Decrypt /etc/ngcp-config-crypted.tgz.gpg and restore configuration files, doing the reverse operation of the encrypt option. Note: This feature is only available if the ngcp-ngcpcfg-locker package is installed.
Show uncomitted changes between ngcpcfg’s Git repository and the working tree inside /etc/ngcp-config. Iff the tool doesn’t report anything it means that there are no uncomitted changes. If the --addremove option is specified then new and removed files (iff present) that are not yet (un)registered to the repository will be reported, no further diff actions will be executed then. Note: This option is available since ngcp-ngcpcfg version 0.11.0.
Encrypt /etc/ngcp-config and all resulting configuration files with a user defined password and save the result as /etc/ngcp-config-crypted.tgz.gpg. Note: This feature is only available if the ngcp-ngcpcfg-locker package is installed.
The help options displays ngcpcfg’s help screen and then exits without any further actions.
The initialise option sets up the ngcpcfg framework. This option is automatically executed by the installer for you, so you shouldn’t have to use this option in normal operations mode.
Retrieve modifications from shared storage. Note: This option is available in the High Availability setup only.
Push modifications to shared storage and remote systems. After changes have been pushed to the nodes the build option will be executed on each remote system to rebuild the configuration files (unless the --nobuild has been specified, then the build step will be skipped). If hostname(s) or IP address(es) is given as argument then the changes will be pushed to the shared storage and to the given hosts only. If no host has been specified then the hosts specified in /etc/ngcp-config/systems.cfg are used. Note: This option is available in the High Availability setup only.
The services option executes the service handlers for any modified configuration file(s)/directory.
The status option provides a human readable interface to check the state of the configuration tree. If you are unsure what should be done as next step or if want to check the current state of the configuration tree just invoke ngcpcfg status.
If everything is OK and nothing needs to be done the output should look like:
# ngcpcfg status Checking state of ngcpcfg: OK: has been initialised already (without shared storage) Checking state of configuration files: OK: nothing to commit. Checking state of /etc files OK: nothing to commit.
If the output doesn’t say "OK" just follow the instructions provided by the output of ngcpcfg status.
Further details regarding the ngcpcfg tool are available through man ngcpcfg on the Sipwise Next Generation Platform.