7. Billing Configuration

This chapter describes the steps necessary to rate calls and export rated CDRs (call detail records) to external systems.

7.1. Billing Data Import

Service billing on the NGCP is based on billing profiles, which may be assigned to VoIP accounts and SIP peerings. The design focuses on a simple, yet flexible approach, to support arbitrary dial-plans without introducing administrative overhead for the system administrators. The billing profiles may define a base fee and free time or free money per billing interval. Unused free time or money automatically expires at the end of the billing interval.

Each profile may have call destinations (usually based on E.164 number prefix matching) with configurable fees attached. Call destination fees each support individual intervals and rates, with a different duration and/or rate for the first interval. (e.g.: charge the first minute when the call is opened, then every 30 seconds, or make it independent of the duration at all) It is also possible to specify different durations and/or rates for peak and off-peak hours. Peak time may be specified based on weekdays, with additional support for manually managed dates based on calendar days. The call destinations can finally be grouped for an overview on user’s invoices by specifying a zone in two detail levels. (E.g.: national landline, national mobile, foreign 1, foreign 2, etc.)

7.1.1. Creating Billing Profiles

The first step when setting up billing data is to create a billing profile, which will be the container for all other billing related data. Go to System AdministrationBilling and click on create new billing profile. You will be taken to a web form where you may enter the following parameters (all values except handle and name may be left empty):

  • handle: A unique, permanently fixed string which is used to attach the billing profile to a VoIP account or SIP peering contract.
  • name: A free form string used to identify the billing profile in the Admin Panel. This may be changed at any time.
  • interval charge: A base fee for the billing interval, specifying a monetary amount (represented as a floating point number) in whatever currency you want to use.
  • interval free time: If you want to include free calling time in your billing profile, you may specify the number of seconds that are available every billing interval. See Creating Billing Fees below on how to select destinations which may be called using the free time.
  • interval free cash: Same as for interval free time above, but specifies a monetary amount which may be spent on outgoing calls. This may be used for example to implement a minimum turnover for a contract, by setting the interval charge and interval free cash to the same values.
  • fraud limit: The fraud detection limit (in Cent) for accounts with this billing profile. If the call fees of an account reach this limit within a billing interval, an action can be triggered.
  • fraud lock: a choice of none, foreign, outgoing, incoming, global. Specifies a lock level which will be used to lock the account and his subscribers when fraud limit is exceeded.
  • fraud notify: An email address or comma-separated list of email addresses that will receive notifications for fraud detections.
  • currency: The currency symbol for your currency. Any UTF-8 character may be used and will be printed in web interfaces.
  • VAT rate: The percentage of value added tax for all fees in the billing profile. Currently for informational purpose only and not used further.
  • VAT included: Whether VAT is included in the fees entered in web forms or uploaded to the platform. Currently for informational purpose only and not used further.

7.1.2. Creating Billing Fees

To set up billing fees, go to System AdministrationBilling and select edit fees next to the billing profile you want to configure. Billing fees may be uploaded using a configurable CSV file format, or entered directly via the web interface by selecting create new entry just below Stored Billing Fees. To configure the CSV field order for the file upload, rearrange the entries in the www_adminfees_csvelement_order array in /etc/ngcp-config/config.yml and execute the command ngcpcfg apply. For input via the web interface, just fill in the text fields accordingly. In both cases, the following information may be specified independently for every destination:

  • destination: The destination E.164 prefix, SIP domain (or IP address) or SIP URI. May be a simple string (e.g. 431, sip.sipwise.com or someone@sip.sipwise.com) or a regular expression matching the complete E.164 number, SIP domain or SIP URI (e.g. ^431.*$, ^.*@sip\.sipwise\.com$ or ^someone@sip\.sipwise\.com$). Regular expressions will be stored unmodified, plain strings will be extended exactly as shown in the two examples. The web interface will remove the regular expression prefix and suffix from an entry in the list of store billing fees.
[Important]

The destination needs to be unique for a billing profile. The system will return an error if a destination is specified twice, both for the file upload and the input via the web interface.

[Important]

There are several internal services (vsc, conference, voicebox) which will need a specific destination entry with a domain-based destination. If you don’t want to charge the same (or nothing) for those services, add a fee for destination ^.*@.+\.local$ there. If you want to charge different amounts for those services, break it down into separate fee entries for ^.*@vsc\.local$, ^.*@conference\.local$ and ^.*@voicebox\.local$ with the according fees. NOT CREATING EITHER THE CATCH-ALL FEE OR THE SEPARATE FEES FOR THE .local DOMAIN WILL BREAK YOUR RATING PROCESS!

  • zone: A zone name for a group of destinations. May be used to group destinations for simplified display, e.g. on invoices. (e.g. foreign zone 1)
  • zone detail: A zone name for a more detailed group of destinations. May be used to group destinations for simplified display, e.g. on invoices. (e.g. germany landline)
  • onpeak init rate: The rate for the first rating interval in cent (of whatever currency, represented as a floating point number) per second. Applicable to calls during onpeak hours.
  • onpeak init interval: The duration of the first billing interval, in seconds. Applicable to calls during onpeak hours.
  • onpeak follow rate: The rate for subsequent rating intervals in cent (of whatever currency, represented as a floating point number) per second. Applicable to calls during onpeak hours. Defaults to onpeak init rate.
  • onpeak follow interval: The duration of subsequent billing intervals, in seconds. Applicable to calls during onpeak hours. Defaults to onpeak init interval.
  • offpeak init rate: The rate for the first rating interval in cent (of whatever currency, represented as a floating point number) per second. Applicable to calls during off-peak hours. Defaults to onpeak init rate.
  • offpeak init interval: The duration of the first billing interval, in seconds. Applicable to calls during off-peak hours. Defaults to onpeak init interval.
  • offpeak follow rate: The rate for subsequent rating intervals in cent (of whatever currency, represented as a floating point number) per second. Applicable to calls during off-peak hours. Defaults to offpeak init rate if that one is specified, or to onpeak follow rate otherwise.
  • offpeak follow interval: The duration of subsequent billing intervals, in seconds. Applicable to calls during off-peak hours. Defaults to offpeak init interval if that one is specified, or to onpeak follow interval otherwise.
  • use free time: Specifies whether free time minutes may be used when calling this destination. May be specified in the file upload as 0, n[o], f[alse] and 1, y[es], t[rue] respectively.

7.1.3. Creating Off-Peak Times

To be able to differentiate between on-peak and off-peak calls, the platform stores off-peak times for every billing profile based on weekdays and/or calendar days. To edit the settings for a billing profile, go to System AdministrationBilling and select edit peak times next to the billing profile you want to configure.

To set off-peak times for a weekday, click on edit next to the according weekday. You will be presented with two input fields which both receive a timestamp in the form of hh:mm:ss specifying a time of day for the start and end of the off-peak period. If any of the fields is left empty, the system will automatically insert 00:00:00 (start field) or 23:59:59 (end field). Click on save to store the setting in the database. You may create more than one off-peak period per weekday, and you may edit existing entries using any of the input fields and clicking save next to it. To completely delete a range, just select delete next to the entry.

To specify off-peak ranges based on calendar dates, click on add new date just below Dates. Enter a date in the form of YYYY-MM-DD into the date input field and fill in the start and end timestamps as outlined above. Select save to store the entry, or cancel to close the input form. Existing dates will be listed below, grouped by year. Click on any of the years to view all dates which have been recorded for it. If an entry is added, the corresponding year is expanded automatically. If an already existing date is added, it will overwrite the existing entry.

7.1.4. Fraud Detection and Locking

The NGCP supports a fraud detection feature, which is designed to detect accounts causing unusually high customer costs, and then to perform one of several actions upon those accounts. This feature can be enabled and configured through three billing profile options described in Section 7.1.1, “Creating Billing Profiles”, namely fraud limit, fraud lock and fraud notify.

Once a day, shortly after midnight local time, a background script automatically sifts through all accounts which are linked to a billing profile enabled for fraud detection, and selects those which have caused a higher cost than the fraud limit configured in the billing profile, within the currently active billing interval (e.g. in the current month). It then proceeds to perform at least one of the following actions on those accounts:

  • If fraud lock is set to anything other than none, it will lock the account accordingly (e.g. if fraud lock is set to outgoing, the account will be locked for all outgoing calls).
  • If anything is listed in fraud notify, an email will be sent to the email addresses configured. The email will contain information about which account is affected, which subscribers within that account are affected, the current account balance and the configured fraud limit, and also whether or not the account was locked in accordance with the fraud lock setting. It should be noted that this email is meant for the administrators or accountants etc., and not for the customer.
[Caution]

Accounts that were automatically locked by the fraud detection feature will not be automatically unlocked when the next billing interval starts. This has to be done manually through the administration panel or through the provisioning interface.

[Important]

If fraud detection is configured to only send an email and not lock the affected accounts, it will continue to do so for over-limit accounts every day. The accounts must either be locked in order to stop the emails (only currently active accounts are considered when the script looks for over-limit accounts) or some other action to resolve the conflict must be taken, such as disabling fraud detection for those accounts.

7.2. Billing Data Export

Regular billing data export is done using CSV (comma separated values) files which may be downloaded from the platform using the cdrexport user which has been created during the installation.

7.2.1. File Name Format

In order to be able to easily identify billing files, the file name is constructed by the following fixed-length fields:

<prefix><separator><version><separator><timestamp><separator><sequence number><suffix>

The definition of the specific fields is as follows:

Table 1. CDR export file name format

File name elementLengthDescription

<prefix>

7

A fixed string. Always sipwise.

<separator>

1

A fixed character. Always _.

<version>

3

The format version, a three digit number. Currently 004.

<timestamp>

14

The file creation timestamp in the format YYYYMMDDhhmmss.

<sequence number>

10

A unique 10-digit zero-padded sequence number for quick identification.

<suffix>

4

A fixed string. Always .cdr.


A valid example filename for a billing file created at 2012-03-10 14:30:00 and being the 42nd file exported by the system, is:

sipwise_004_20120310143000_0000000042.cdr

7.2.2. File Format

Each billing file consists of three parts: one header line, zero to 5000 body lines and one trailer line.

7.2.2.1. File Header Format

The billing file header is one single line, which is constructed by the following fields:

<version>,<number of records>

The definition of the specific fields is as follows:

Table 2. CDR export file header line format

Body ElementLengthTypeDescription

<version>

3

zero-padded uint

The format version. Currently 004.

<number of records>

4

zero-padded uint

The number of body lines contained in the file.


A valid example for a Header is:

004,0738
7.2.2.2. File Body Format

The body consists of a minimum of zero and a maximum of 5000 lines. Each line holds one call detail record in CSV format and is constructed by the following fields, all of them enclosed in single quotes:

Table 3. CDR export file body line format

Body ElementLengthTypeDescription

<id>

1-10

uint

Internal CDR id.

<update_time>

19

timestamp

Timestamp of last modification.

<source_user_id>

36

string

Internal UUID of calling party subscriber.

<source_provider_id>

1-255

string

Internal ID of calling party provider.

<source_ext_subscriber_id>

0-255

string

External ID of calling party subscriber.

<source_subscriber_id>

1-10

uint

Internal ID of calling party subscriber.

<source_ext_account_id>

0-255

string

External ID of calling party VoIP account.

<source_account_id>

1-10

uint

Interal ID of calling party VoIP account.

<source_user>

1-255

string

SIP username of calling party.

<source_domain>

1-255

string

SIP domain of calling party.

<source_cli>

1-64

string

CLI of calling party in E.164 format.

<source_clir>

1

uint

1 for calls with CLIR, 0 otherwise.

<destination_user_id>

1 / 36

string

Internal UUID of called party subscriber or 0 if callee is not local.

<destination_provider_id>

1-255

string

Internal ID of called party provider.

<dest_ext_subscriber_id>

0-255

string

External ID of called party subscriber.

<dest_subscriber_id>

1-10

uint

Internal ID of called party subscriber.

<dest_ext_account_id>

0-255

string

External ID of called party VoIP account.

<destination_account_id>

1-10

uint

Interal ID of called party VoIP account.

<destination_user>

1-255

string

Final SIP username of called party.

<destination_domain>

1-255

string

Final SIP domain of called party.

<destination_user_in>

1-255

string

Incoming SIP username of called party.

<destination_domain_in>

1-255

string

Incoming SIP domain of called party.

<peer_auth_user>

0-255

string

User to authenticate towards peer.

<peer_auth_realm>

0-255

string

Realm to authenticate towards peer.

<call_type>

3-4

string

The type of the call - one of:

call: normal call

cfu: call forward unconditional

cft: call forward timeout

cfb: call forward busy

cfna: call forward no answer

<call_status>

2-7

string

The final call status - one of:

ok: successful call

busy: callee busy

noanswer: no answer from callee

cancel: cancel from caller

offline callee offline

timeout: no reply from callee

other: unspecified, see <call_code> for details

<call_code>

3

uint

The final SIP status code.

<init_time>

23

timestamp

Timestamp of call initiation (invite received from caller). Seconds include fractional part (3 decimals).

<start_time>

23

timestamp

Timestamp of call establishment (final response received from callee). Seconds include fractional part (3 decimals).

<duration>

4-11

fixed precision

Length of call (beginning at start_time) in seconds with 3 decimals.

<call_id>

1-255

string

The SIP call-id.

<rating_status>

2-7

string

The internal rating status - one of:

unrated: not rated

ok: successfully rated

failed: error while rating

Currently always ok or unrated, depending on whether rating is enabled or not.

<rated_at>

0 / 19

timestamp

Timestamp of rating or empty if not rated.

<carrier_cost>

4-11

fixed precision

The carrier termination cost or empty if not rated. In cent with two decimals.

<customer_cost>

4-11

fixed precision

The customer cost or empty if not rated. In cent with two decimals.

<carrier_zone>

0-127

string

The carrier billing zone or empty if not rated.

<customer_zone>

0-127

string

The customer billing zone or empty if not rated.

<carrier_destination>

0-127

string

The carrier billing destination or empty if not rated.

<customer_destination>

0-127

string

The customer billing destination or empty if not rated.

<dialed_digits>

1-255

string

The user-part of the SIP Request URI as received by the soft-switch.

<carrier_free_time>

1-10

uint

The number of free time seconds used on carrier side or empty if not rated.

<customer_free_time>

1-10

uint

The number of free time seconds used from the customer’s account balance or empty if not rated.

<line_terminator>

1

string

A fixed character. Always \n (special char LF - ASCII 0x0A).


A valid example of one body line of a rated CDR is (line breaks added for clarity):

  '2055','2007-11-07 11:36:49','6b6977f9-6125-4339-a82c-3c5af04652d2','1',
's12345','42','c2468','23','test','sipwise.com','43720456700','0','0','3',,
'0',,'0','4315551234','192.168.101.17','4315551234','sipwise.com',,,'call',
'ok','200','2007-11-05 16:17:33.517','2007-11-05 16:17:37.641','74.731',
'7F2A3EA-47F348E84@192.168.101.9','ok','2007-11-07 11:36:49','9.25','16.03',
'national landline','national landline','landline vienna','landline vienna',
'015551234','0','0'
7.2.2.3. File Trailer Format

The billing file trailer is one single line, which is constructed by the following fields:

<md5 sum>

The <md5 sum> is a 32 character hexadecimal MD5 hash of the Header and Body.

To validate the billing file, one must remove the Trailer before computing the MD5 sum of the file. An example bash script to validate the integrity of the file is given below:

#!/bin/sh

error() { echo $@; exit 1; }
test -n "$1" || error "Usage: $0 <cdr-file>"
test -f "$1" || error "File '$1' not found"

TMPFILE="/tmp/$(basename "$1")"
MD5="$(sed -rn '$ s/^([a-z0-9]{32}).*$/\1/i p' "$1")  $TMPFILE"
sed '$d' "$1" > "$TMPFILE"
echo "$MD5" | md5sum -c -
rm -f "$TMPFILE"

Given the script is located in cdr-md5.sh and the CDR-file is sipwise_001_20071110123000_0000000004.cdr, the output of the integrity check for an intact CDR file would be:

$ ./cdr-md5.sh sipwise_001_20071110123000_0000000004.cdr
/tmp/sipwise_001_20071110123000_0000000004.cdr: OK

If the file has been altered during transmission, the output of the integrity check would be:

$ ./cdr-md5.sh sipwise_001_20071110123000_0000000004.cdr
/tmp/sipwise_001_20071110123000_0000000004.cdr: FAILED
md5sum: WARNING: 1 of 1 computed checksum did NOT match

7.2.3. File Transfer

Billing files are created twice per hour at minutes 25 and 55 and are stored in the home directory of the cdrexport user. If the amount of records within the transmission interval exceeds the threshold of 5000 records per file, multiple billing files are created. If no billing records are found for an interval, a billing file without body data is constructed for easy detection of lost billing files on the 3rd party side.

CDR files are fetched by a 3rd party billing system using SFTP or SCP with either public key or password authentication using the username cdrexport. If public key authentication is chosen, the public key file has to be stored in the file ~/.ssh/authorized_keys2 below the home directory of the cdrexport user. Otherwise, a password has to be set for the user.

The 3rd party billing system is responsible for deleting CDR files after fetching them.

[Note]

The cdrexport user is kept in a jailed environment on the system, so it has only access to a very limited set of commandline utilities.