Why might rdmd not be running all unit tests?

John Doucette

Unfortunately I cannot seem to reproduce this behavior in a minimal working example, so this may be too vague. However, I can at least state what is not causing the behavior.

I have a D module containing several classes with unit tests, with structure similar to:

import std.stdio;

class c1{
        int c1func(int blah){ return blah; }
        unittest{
                writeln("Testing c1 func.");
                stdout.flush();
        }
}

class c2(T:c3) : c1{
        private static int c2func(int blah){ return blah; }
        unittest{
                writeln("Testing c2 func.");
                stdout.flush();
        }

        int c2fun2(int blah){
                T tmp = new T();
                return tmp.c3fun(blah);
        }
}

class c3{
        abstract int c3fun(int blah);
}

class c4 : c3{
        override int c3fun(int blah){ return blah; }
}

unittest{
        writeln("Testing class c1.");
        stdout.flush();
        c1 myc2 = new c2!(c4)();
}

My expectation is that a call of the form:

rdmd --main -unittest tmp.d

will produce output

Testing c1 func.
Testing class c1.
Testing c2 func.

and it does (c2 func's unit tests are not run until the time of instantiation).

However, in my similar, but much longer D module, no unit tests for the template class c2's counterpart are run (I've put print statements identical to the one's here throughout my other unit tests to be sure).

This is a bit worrisome, as in fact, I'd been relying on the correctness of these other unit tests for some time. Further, rdmd happily picks up on syntax errors in the unit test code, despite never running said code.

So here's what I can rule out:

  • The unit tests are being read by rdmd.
  • rdmd is actually being run against the most recent version of the file.
  • rdmd apparently is happy to run unit tests for implementations of template classes, even with specializations of abstract classes for their arguments, as long as those template classes are actually instantiated.
  • The functions in my template class (akin to c2 in the example code) are actually getting called during the main unit test block at the bottom, so the template class has definitely been instantiated.
  • rdmd itself is not at fault, as compiling using dmd and the same flags also fails to run these unit tests.

Any ideas about what could be going wrong here? I'd provide a MWE if I could, but every version of one I've written seems to work fine!

John Doucette

It seems that this results from the way that rdmd orders the unit tests, which is not in the way I'd expected.

It appears that RDMD calls the unit tests for an instantiated template class not at the time of instantiation, but instead immediately after the unit test in which the class is instantiated has completed.

Here's a MWE:

import std.stdio;

class c1{
        int c1func(int blah){ return blah; }
        unittest{
                writeln("Testing c1 func.");
                stdout.flush();
        }
}

class c2(T:c3) : c1{
        private static int c2func(int blah){ return blah; }
        unittest{
                writeln("Testing c2 func.");
                stdout.flush();
        }

        int c2fun2(int blah){
                T tmp = new T();
                return tmp.c3fun(blah);
        }
}

class c3{
        abstract int c3fun(int blah);
}

class c4 : c3{
        override int c3fun(int blah){ return blah; }
}

unittest{
        writeln("Testing class c1.");
        stdout.flush();
        c1 myc2 = new c2!(c4)();
        assert(1==0);
}

In this case, the unit test at the bottom will fail (because of assert(1==0)), and so the template class's unit tests are never run.

This poses a major problem when using template classes together with unit tests. If your template class fails is broken in some way that causes the unit test where it was instantiated to fail, then its own unit tests are never run. In the case of silent failure (a silent exit was triggered in my code by bugs in the template class), this will manifest as the template class's tests never running.

Summary: DMD usually just runs unit tests in the order that they appear. However, since template classes don't exist without some completion of the template, unit tests inside a template class are not run in the order that they appear. Instead, they are run (in order), immediately after the first unit test where the template class is instantiated.

Workaround: adding a separate unit test immediately below the template class that just instantiates it with all the types one wants to test will cause the unit tests to be run, in the correct order. However, a seemingly better policy would be for to run those unit tests before the test block in which the instantiation occurs!

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

From Java

Visual Studio 15.8.1 not running MS unit tests

From Dev

Why does Travis fail to find rospy when running unit tests?

From Dev

Mocha unit tests running with Karma - done() is not defined

From Dev

Saving unit test results after running tests

From Dev

MSTest: running unit tests in 64 bit mode

From Dev

Xcode 5: Unit Tests not running

From Dev

OpenCover without running unit tests

From Dev

Exception running unit tests with Fakes on build server

From Dev

Running Unit and Integration Tests in IntelliJ

From Dev

Error when running unit tests in Xcode 5.0

From Dev

Why are my unit tests not working?

From Dev

Running unit-tests using PyDev

From Dev

running a set of unit tests for each 'implementation'

From Dev

C++ Unit Test fails when running all tests or after a certain test

From Dev

Running spock unit tests with Maven

From Dev

Running Boost unit tests on different processes

From Dev

Running unit tests in Python with a caching decorator

From Dev

Switching branches while running python unit tests

From Dev

Resharper Unit Tests not running

From Dev

Running unit tests before each build in Gradle

From Dev

Why aren't my unit tests running in the project output folder?

From Dev

Running Spark Unit Tests in IntelliJ

From Dev

When multiple unit tests copy the same file, running all unit tests fail

From Dev

Running QtQuickTest unit tests

From Dev

Running Android unit tests with Spock framework

From Dev

MSTest: running unit tests in 64 bit mode

From Dev

Running unit-tests using PyDev

From Dev

Running all tests with Quickbuild

From Dev

When multiple unit tests copy the same file, running all unit tests fail

Related Related

HotTag

Archive