Ebee Unit Tests C (Unity)

For unit tests in Ansi-C we are using the test framework Unity, information can be found here http://www.throwtheswitch.org . The tests themselves using the typical assertion mechanism as in many other unit test frameworks. There is the famous AAA(triple A) pattern which can be found for example here https://methodpoet.com/aaa-in-unit-testing/ .

Unity is the unit test framework itself with various assertion mechanisms, a good description is here https://github.com/ThrowTheSwitch/Unity.

This is the unit test build management system, an introduction can be found in http://www.throwtheswitch.org/ceedling . Normally a developer will not get in touch with the installation or configuration of it except for adding needed paths for include files. The system is already set up in our meta projects https://gitlab.com/ebee_smart/linux4sam/linux4sam-meta and https://gitlab.com/ebee_smart/ebee-controller-meta.

For functions which are not directly addressed there is the mocking framework CMock https://github.com/ThrowTheSwitch/CMock/blob/master/docs/CMock_Summary.md . With the help of it functions can be mocked, that means we can “simulate” the behavior of them. It is possible to let return specific return codes or completely simulate the behavior. It is also possible to check if a specific function was called and also check if the parameters were set properly.

In the meta projects there are already prebuilt make targets.

`make run-unit-tests` can be used to let run all unit tests. If they have to be compiled it will also done with this make target. The libebee_common will also be built with this target if it was not built before. One dedicated testsuite could be selected for test by option “UNITTEST_TEST=<testsuite>” on make call. (Testsuite means the basename of each test file or “libebee_common” as a whole).

`make wipe-unit-tests` can be used to remove all files generated by the unit test build management.

`make unit-test-shell` starts the Docker container and set some environment variables to just call the ceedling unit test build system directly. For example it is possible to just call/build a specific test you can then call inside the shell `ceedling test:voltage` for performing all unit tests of the file `test/system_monitoring/monitoring/test_voltage.c`.

Specific tests can also be called from outside the container. Just call `CEEDLING_CMD=test:voltage make run-unit-tests`. For performing only tests which were modified since the last test run you can call `CEEDLING_CMD=test:delta make run-unit-tests`.

It is also possible to build and run unit tests without the high level meta project included make targets. These targets are only available in newer branches >= 5.30. For older maintenance branches e.g. 5.20 here is a general guide.

First it is needed to compile the libebee_common as a shared object library. This can be done with the meta project as root folder:

1. cd controller_software
2. BUILDROOT=../buildroot test/support/build_ebee_lib.sh

The file libebee_common.so should then be located in libebee_common/ .

1. cd to the meta project main directory
2. start the container with
   

   make run-container

3. cd controller_software

4. start the unit test run with

   BUILDROOT=../buildroot CC=gcc ceedling

1. start the container with

   make run-container

2. cd controller_software

3. start the unit test run with

   BUILDROOT=../buildroot CC=gcc ceedling test:delta

1. start the container with

   make run-container

2. cd controller_software

3. start the unit test run with

   BUILDROOT=../buildroot CC=gcc ceedling test:<test module>

   
   e.g.

   BUILDROOT=../buildroot CC=gcc ceedling test:voltage

In Visual Studio Code it is possible to let run the modified unit tests. There is the build task `5 ceedling test:delta`. You can call it with `Terminal→Run Build Task…→5 ceedling test:delta`. There is also the shortcut `Ctrl+Shift+B`.

For each unit test module built by ceedling an according binary will be created. The libebee_common is needed as a shared object libebee_common.so. If it is not present in ./libebee_common or outdated it can be built with BUILDROOT=./buildroot controller_software/test/support/build_ebee_lib.sh from the meta project.

If for example the test test_voltage should be debugged it has to be built first from the meta project with e.g.

CEEDLING_CMD=test:voltage make run-unit-tests

The binary is located in controller_software/build/test/out/ . If the test module test_voltage was built the binary is then controller_software/build/test/out/test_voltage.out.

If there are errors for missing libebee_common.so when executing the binary the LD_LIBRARY_PATH has to be extended by the path where the libebee_common.so is located, e.g. a call from controller_software/build/test/out/ can look like LD_LIBRARY_PATH=$LD_LIBRARY_PATH:../../libebee_common test/out/test_voltage.out .

Now the binary can be debugged with e.g. gdb.

If VSCode is used there is a pre-built launch target in .vscode/launch.json

{
    "name": "(gdb) ceedling unit test",
    "type": "cppdbg",
    "request": "launch",
    "program": "${workspaceFolder}/build/test/out/test_meter_measurands.out",
    "args": [],
    "stopAtEntry": false,
    "cwd": "${workspaceFolder}",
    "environment": [],
    "MIMode": "gdb",
    "miDebuggerPath": "/usr/bin/gdb",
    "setupCommands": [
        {
            "description": "Enable pretty-printing for gdb",
            "text": "-enable-pretty-printing",
            "ignoreFailures": true
        }
    ]
}

The program name should be then modified in “program”.

If a new test file should be added then just add the according test file with the prefix “test_”. It is strongly recommended that the same file structure will be used.

If you want to add unit tests for the file "src/system_monitoring/monitoring/uptime.c" you can create an according test file in "test/system_monitoring/monitoring/" with the name "test_uptime.c".

For test modules it is mandatory to include the header "unity.h".

#include "unity.h"

For adding the file to test in the test module it can be either done by adding the according header but in almost all unit test modules we just include the c file itself. It has the advantage to also have influence on static variables and also to call static functions (if it is needed) out of the unit tests. Here the example:

#include "unity.h"
#include "system_monitoring/monitoring/uptime.c"

If there are additional headers used their paths have to be added in the file `project.yaml` in the section `:source`.

Alternatively you can use ceedling within the `make unit-test-shell` environment to create the file for you by calling:

alex@cpbuild:~$ ceedling module:create[system_monitoring/monitoring/uptime]
File src/system_monitoring/monitoring/uptime.c already exists!
File src/system_monitoring/monitoring/uptime.h already exists!
File test/system_monitoring/monitoring/test_uptime.c created
Generate Complete

This will generate a test file based on a template

It is possible to call a setup and/or a teardown function which will be called for each unit test, in the example there is just a log message. The setup/teardown should only be used if needed. A good example would be to register/unregister PStore parameter to create an own parameter store. There are examples in some unit test modules, e.g. here "test/helpers/test_branding.c".

In our example (just a log message written)

#include "unity.h"
#include "system_monitoring/monitoring/uptime.c"
#include <log/log.h>

void setUp(void)
{
    LOG_INFO_FN("set up");
}
void tearDown(void)
{
    LOG_INFO_FN("tear down");
}

For adding tests it is mandatory to use the prefix "test_" in each unit test method, for example

#include "unity.h"
#include "system_monitoring/monitoring/uptime.c"

void test_monitoring_uptime_timer_expired_return_correct_value(void)
{
}

Now we can add a unit test according to the AAA pattern.

#include "unity.h"
#include "system_monitoring/monitoring/uptime.c"
void test_monitoring_uptime_timer_expired_return_correct_value(void)
{
    // arrange
    uptime_timer_ev = 123;

    // act
    int ret = monitoring_uptime_timer_expired();

    // assert
    TEST_ASSERT_EQUAL(0, ret);

    // arrange
    uptime_timer_ev = 0;

    // act
    ret = monitoring_uptime_timer_expired();

    // assert
    TEST_ASSERT_EQUAL(1, ret);
}

In the example unit test there are 2 assertions, one for the case the timer was expired, the other for the not expired case. It is also possible to have 2 separate unit tests, for example

#include "unity.h"
#include "system_monitoring/monitoring/uptime.c"

void test_monitoring_uptime_timer_expired_return_1_if_expired(void)
{
    // arrange
    uptime_timer_ev = 0;

    // act
    int ret = monitoring_uptime_timer_expired();

    // assert
    TEST_ASSERT_EQUAL(1, ret);
}

void test_monitoring_uptime_timer_expired_return_0_if_not_expired(void)
{
    // arrange
    uptime_timer_ev = 123;

    // act
    int ret = monitoring_uptime_timer_expired();

    // assert
    TEST_ASSERT_EQUAL(0, ret);
}

If the behavior of functions has to be simulated then mocking is the choice. For example we want to test the behavior of `uptime_timer_ev`

void test_uptime_timer_cb_sets_uptime_timer_ev_to_zero(void)
{
    // some arrangements
    uptime_timer_cb(NULL);

    // some assertions
}

If the test will be compiled then some linker errors will appear like

ebee-controller-meta-new-buildroot/controller_software/src/system_monitoring/monitoring/uptime.c:43: undefined reference to `ocpp_is_ready_for_transaction_notified'

Those errors come from the linker because the used functions are not defined because the source files were not compiled and linked. For this reason we can define a mocking function and also let the mock define the behavior to test different code paths. This should be done by writing a mock for every used function. The mocks will be generated when the according header files will also be included in the test module, just extended with the prefix 'mock_\<include file>'.

#include "unity.h"

#include "system_monitoring/monitoring/uptime.c"

#include "ocpp_logic/ocpp_hooks/mock_ocpp_on_ready_for_transaction_hook.h"
#include "meter/mock_meter.h"
#include "error_handling/mock_ebee_error_handling.h"
#include "ocpp_logic/ocpp_connector/mock_ocpp_connector.h"
#include "eevt/mock_eevt.h"

For all added mock headers their paths have to be added in the file `project.yaml` in the section `:source` if they are not present there yet.

• src/ocpp_logic/ocpp_hooks
• src/meter
• src/error_handling
• src/ocpp_logic/ocpp_connector
• src/eevt

Let's write a unit test for the function `uptime_timer_cb`. There are several function calls

static void uptime_timer_cb(void *arg __attribute__((unused)))
{
    LOG_INFO("System uptime is %d minutes", UPTIME_TIMOUT_MIN);

    uptime_timer_ev = 0;

    if (! ocpp_is_ready_for_transaction_notified() && meter_is_ready_for_billing()
        && ! ocpp_connector_is_unavailable_or_faulted(OCPP_LOCAL_CONNECTOR)
        && error_is_charging_allowed(OCPP_LOCAL_CONNECTOR)) {
        eevt_trigger(EEVT_OCPP_READY_FOR_TRANSACTION, &(bool){true});
    }
}

For example we want to check if the callback calls `eevt_trigger` with the correct arguments. There are some conditions:

1. `ocpp_is_ready_for_transaction_notified` should return `false`
2. `meter_is_ready_for_billing` should return `1`
3. `ocpp_connector_is_unavailable_or_faulted` should be called with the parameter `OCPP_LOCAL_CONNECTOR` and should return `0`
4. `error_is_charging_allowed` should be called with the parameter `OCPP_LOCAL_CONNECTOR` and should return `1` `eevt_trigger` should be called with the parameters `EEVT_OCPP_READY_FOR_TRANSACTION` and `&(bool) {true}`.

The conditions are met easily. There are parameters expected in some functions and some defined return codes to be set.

ocpp_is_ready_for_transaction_notified_ExpectAndReturn(false);
meter_is_ready_for_billing_ExpectAndReturn(1);
ocpp_connector_is_unavailable_or_faulted_ExpectAndReturn(OCPP_LOCAL_CONNECTOR, 0);
error_is_charging_allowed_ExpectAndReturn(OCPP_LOCAL_CONNECTOR, 1);

The mocks are created by just adding the postfix `_ExpectAndReturn`.

The first parameters are the expected values. Internally the mocked function will perform an assertion. It is also possible to ignore the parameters. Then the postfix `_IgnoreAndReturn` should be used. The last parameter is the return code the mock should return. In most cases this easy mock creation will work. Sometimes it is more complicated where for example pointers are used in the functions to be mocked. Then it is also possible to do this by just defining a complete custom behavior by just writing a stub. As you can see the function `eevt_trigger` expects the second parameter `&(bool) {true}`, so the address of a variable holding the boolean value `true`.

For defining a stub as a separate function you can just use the function prototype and as the last parameter you have to add the number of function calls as an integer. This can then also be used to define a behavior depending of an iteration or similar. A stub could look like this:

static int (**eevt_trigger_stub(eevt_type_t eevt_type, const void *data, int num_calls))()
{
    TEST_ASSERT_EQUAL(EEVT_OCPP_READY_FOR_TRANSACTION, eevt_type);
    bool data_as_bool = *((bool *)data);

    TEST_ASSERT_TRUE(data_as_bool);
    return 0;
};

For using the stub it can be done by

eevt_trigger_StubWithCallback(eevt_trigger_stub);

An alternative way and a possibility to mock LIBC functions too, is to define the function in your test file locally again. The linker will choose this as the implementation and you could define the functionality and return values by your own.

BUT BE WARNED: Keep the mocks of common known functions (LIBC, …) really local (with '__attribute__ ((visibility ("hidden")));') in your implementation and be carefully that the mocked common known functions not influence 3rd party libs. Please see this example:

Function to test is:

int ssl_comm_read_from_pkcs_file(const char *pkcs_file_name, X509 **x509, EVP_PKEY **pkey,
                                 STACK_OF(X509) * *cas)
{
    int ret = -1;

    if (! pkcs_file_name || ! x509 || ! pkey) {
        return -EFAULT;
    }

    FILE *fd = fopen(pkcs_file_name, "r");
    if (! fd) {
        LOG_WARN_FN("Could not open PKCS12 container: %s", pkcs_file_name);
    } else {
        PKCS12 *pkcs12 = d2i_PKCS12_fp(fd, NULL);
        fclose(fd);
        if (! pkcs12) {
            LOG_WARN_FN("Could not decode PKCS12 container");
        } else {
            int ret1 = PKCS12_parse(pkcs12, ssl_comm_pw, pkey, x509, cas);
            PKCS12_free(pkcs12);
            if (ret1 != 1) {
                LOG_WARN_FN("Could not parse PKCS12 container");
            } else {
                ret = 0; // success
            }
        }
    }
    return ret;
}

For a successful test, one have to mock fopen(), d2i_PKCS12_fp() and PKCS12_parse().

Define the function mocks together with a static variable for the return value to set before executing the test:

/* function mocks definition */
static FILE *fopen_ret;
FILE *fopen(const char *filename, const char *mode) __attribute__ ((visibility ("hidden")));
FILE *fopen(const char *filename, const char *mode)
{
    return fopen_ret;
}

int fclose(FILE *fp) __attribute__ ((visibility ("hidden")));
int fclose(FILE *fp)
{
    return 0;
}

static void *d2i_PKCS12_fp_ret;
PKCS12 *d2i_PKCS12_fp(FILE *fp, PKCS12 **p12)
{
    return d2i_PKCS12_fp_ret;
}

void PKCS12_free(PKCS12 *p12)
{
    return;
}

/** @return 0 ... error, 1 ... success */
static int PKCS12_parse_ret;
int PKCS12_parse(PKCS12 *p12, const char *pw, EVP_PKEY **pkey, X509 **cert, STACK_OF(X509) * *ca)
{
    return PKCS12_parse_ret;
}

And the test using it will set the expected return value before calling the function to test than uses the mocked functions above:

void test_ssl_comm_read_from_pkcs_file(void)
{
    X509 *x509 = NULL;
    EVP_PKEY *pkey = NULL;
    /* set expected mock return values to failure */
    fopen_ret = NULL;
    d2i_PKCS12_fp_ret = NULL;
    PKCS12_parse_ret = 0;

    /* case 2: file not found, fopen() failed */
    ret = ssl_comm_read_from_pkcs_file("dummy", &x509, &pkey, NULL);
    TEST_ASSERT_EQUAL_INT(-1, ret);
    TEST_ASSERT_NULL(x509);
    TEST_ASSERT_NULL(pkey);

    /* case 3: fopen() succeed, but d2i_PKCS12_fp() failed */
    fopen_ret = (FILE *)1; /* change mock return value to success */
    ret = ssl_comm_read_from_pkcs_file("dummy", &x509, &pkey, NULL);
    TEST_ASSERT_EQUAL_INT(-1, ret);
    TEST_ASSERT_NULL(x509);
    TEST_ASSERT_NULL(pkey);

    /* case 4: fopen() and d2i_PKCS12_fp() succeed, but PKCS12_parse() failed */
    fopen_ret = (FILE *)1;           /*  change mock return value to success */
    d2i_PKCS12_fp_ret = (PKCS12 *)1; /*  change mock return value to success */

    ret = ssl_comm_read_from_pkcs_file("dummy", &x509, &pkey, NULL);
    TEST_ASSERT_EQUAL_INT(-1, ret);
    TEST_ASSERT_NULL(x509);
    TEST_ASSERT_NULL(pkey);

    /* case 5: fopen(), d2i_PKCS12_fp() and PKCS12_parse() succeed */
    fopen_ret
        = (FILE
               *)1; /*  change mock return value to success (not rellay needed, for explaination) */
    d2i_PKCS12_fp_ret = (PKCS12 *)1; /*  change mock return value to success */
    PKCS12_parse_ret = 1;            /*  change mock return value to success */

    ret = ssl_comm_read_from_pkcs_file("dummy", &x509, &pkey, NULL);
    TEST_ASSERT_EQUAL_INT(0, ret);
}

#include "unity.h"

#include "system_monitoring/monitoring/uptime.c"

#include "ocpp_logic/ocpp_hooks/mock_ocpp_on_ready_for_transaction_hook.h"
#include "meter/mock_meter.h"
#include "error_handling/mock_ebee_error_handling.h"
#include "ocpp_logic/ocpp_connector/mock_ocpp_connector.h"
#include "eevt/mock_eevt.h"

void test_monitoring_uptime_timer_expired_return_1_if_expired(void)

    // arrange
    uptime_timer_ev = 0;

// act
int ret = monitoring_uptime_timer_expired();

// assert
TEST_ASSERT_EQUAL(1, ret);
}

void test_monitoring_uptime_timer_expired_return_0_if_not_expired(void)
{
    // arrange
    uptime_timer_ev = 123;

    // act
    int ret = monitoring_uptime_timer_expired();

    // assert
    TEST_ASSERT_EQUAL(0, ret);
}

int (**eevt_trigger_stub(eevt_type_t eevt_type, const void *data, int num_calls))()
{
    TEST_ASSERT_EQUAL(EEVT_OCPP_READY_FOR_TRANSACTION, eevt_type);
    bool data_as_bool = *((bool *)data);

    TEST_ASSERT_TRUE(data_as_bool);
    return 0;
};

void test_uptime_timer_cb_calls_correct_eevt_trigger(void)
{
    // arrange
    uptime_timer_ev = 1;

    ocpp_is_ready_for_transaction_notified_ExpectAndReturn(false);
    meter_is_ready_for_billing_ExpectAndReturn(1);
    ocpp_connector_is_unavailable_or_faulted_ExpectAndReturn(OCPP_LOCAL_CONNECTOR, 0);
    error_is_charging_allowed_ExpectAndReturn(OCPP_LOCAL_CONNECTOR, 1);
    eevt_trigger_StubWithCallback(eevt_trigger_stub);

    // act
    uptime_timer_cb(NULL);

    // assert
    TEST_ASSERT_EQUAL(0, uptime_timer_ev);
}

If you want to create a unit test for an module that uses functions from an Cpp module, you have to create an mock header for the Cpp module, extended with the postfix mock. Like in dlm_charts_db_mock.h.

#ifndef DB_DLM_CHARTS_DB_MOCK_H_
#define DB_DLM_CHARTS_DB_MOCK_H_

void dlm_charts_db_upgrade(int version) {(void)version;}
void dlm_charts_db_reduce_table_size(void) {}
void dlm_charts_add_slave_entry(int slave_id) {(void)slave_id;}
int  dlm_charts_db_get_count(void) {return 0;}

#endif /* DB_DLM_CHARTS_DB_MOCK_H_ */

That header you have to include instead of the prefix mock header.

#include "unity.h"

#define HOSTSIMULATION

#include "db.c"

#include "sqlite3.h"

#include "mock_ebee_version.h"
#include "mock_transaction_db.h"
#include "mock_dlm_slaves_db.h"
// #include "mock_dlm_charts_db.h"
#include "db/dlm_charts_db_mock.h"
#include "mock_key_value_store_db.h"
#include "config/parameters/mock_database_parameters.h"