This chapter is from the book
This chapter is from the book
This chapter is from the book
Test Spy
How do we implement Behavior Verification?
How can we verify logic independently when it has indirect outputs to other software components?
We use a Test Double to capture the indirect output calls made to another component by the SUT for later verification by the test.
In many circumstances, the environment or context in which the SUT operates very much influences the behavior of the SUT. To get adequate visibility of the indirect outputs of the SUT, we may have to replace some of the context with something we can use to capture these outputs of the SUT.
Use of aTest Spyis a simple and intuitive way to implementBehavior Verificationvia an observation point that exposes the indirect outputs of the SUT so they can be verified.
How It Works
Before we exercise the SUT, we install aTest Spyas a stand-in for a DOC used by the SUT. TheTest Spyis designed to act as an observation point by recording the method calls made to it by the SUT as it is exercised. During the result verification phase, the test compares the actual values passed to theTest Spyby the SUT with the values expected by the test.
When to Use It
A key indication for using aTest Spyis having anUntested Requirement(seeProduction Bugs) caused by an inability to observe the side effects of invoking methods on the SUT.Test Spiesare a natural and intuitive way to extend the existing tests to cover these indirect outputs because the calls to theAssertion Methodsare invoked by the test after the SUT has been exercised just like in “normal” tests. TheTest Spymerely acts as the observation point that gives theTest Methodaccess to the values recorded during the SUT execution.
We should use aTest Spyin the following circumstances:
- We are verifying the indirect outputs of the SUT and wecannotpredict the values of all attributes of the interactions with the SUT ahead of time.
- We want the assertions to be visible in the test and we don’t think the way in which theMock Objectexpectations are established is sufficiently intent-revealing.
- We want the assertions to be visible in the test and we don’t think the way in which theMock Objectexpectations are established is sufficiently intent-revealing.
- Our test requires test-specific equality (so we cannot use the standard definition of equality as implemented in the SUT)andwe are using tools that generate theMock Objectbut do not give us control over theAssertion Methodsbeing called.
- A failed assertion cannot be reported effectively back to theTest Runner．This might occur if the SUT is running inside a container that catches all exceptions and makes it difficult to report the results or if the logic of the SUT runs in a different thread or process from the test that invokes it. (Both of these cases really beg refactoring to allow us to test the SUT logic directly, but that is the subject of another chapter.)
- We would like to have access to all the outgoing calls of the SUT before making any assertions on them.
If none of these criteria apply, we may want to consider using aMock Object．If we are trying to addressUntested Code(seeProduction Bugs) by controlling the indirect inputs of the SUT, a simpleTest Stubmay be all we need.
Unlike aMock Object,aTest Spydoes not fail the test at the first deviation from the expected behavior. Thus our tests will be able to include more detailed diagnostic information in theAssertion Messagebased on information gathered after aMock Objectwould have failed the test. At the point of test failure, however, only the information within theTest Methoditself is available to be used in the calls to theAssertion Methods．如果我们需要包括的信息访问ible only while the SUT is being exercised, either we must explicitly capture it within ourTest Spyor we must use aMock Object．
Of course, we won’t be able to use anyTest Doubles除非SUT及其ments some form of substitutable dependency.
Implementation Notes
TheTest Spyitself can be built as aHard-Coded Test Doubleor as aConfigurable Test Double．Because detailed examples appear in the discussion of those patterns, only a quick summary is provided here. Likewise, we can use any of the substitutable dependency patterns to install theTest Spybeforewe exercise the SUT.
The key characteristic in how a test uses aTest Spyrelates to the fact that assertions are made from within theTest Method．Therefore, the test must recover the indirect outputs captured by theTest Spybefore it can make its assertions, which can be done in several ways.
Variation: Retrieval Interface
We can define theTest Spyas a separate class with aRetrieval Interfacethat exposes the recorded information. TheTest Methodinstalls theTest Spyinstead of the normal DOC as part of the fixture setup phase of the test. After the test has exercised the SUT, it uses theRetrieval Interfaceto retrieve the actual indirect outputs of the SUT from theTest Spyand then callsAssertion Methodswith those outputs as arguments.
Variation: Self Shunt
We can collapse theTest Spyand theTestcase Classinto a single object called aSelf Shunt．TheTest Methodinstalls itself, theTestcase Object, as the DOC into the SUT. Whenever the SUT delegates to the DOC, it is actually calling methods on theTestcase Object,这implements the methods by saving the actual values into instance variables that can be accessed by theTest Method．The methods could also make assertions in theTest Spymethods, in which case theSelf Shuntis a variation on aMock Objectrather than aTest Spy．In statically typed languages, theTestcase Classmust implement the outgoing interface (the observation point) on which the SUT depends so that theTestcase Class istype-compatible with the variables that are used to hold the DOC.
Variation: Inner Test Double
A popular way to implement theTest Spyas aHard-Coded Test Doubleis to code it as ananonymous inner classorblock closurewithin theTest Methodand to have this class orblocksave the actual values into instance or local variables that are accessible by theTest Method．This variation is really another way to implement aSelf Shunt(seeHard-Coded Test Double).
Variation: Indirect Output Registry
Yet another possibility is to have theTest Spystore the actual parameters in a well-known place where theTest Methodcan access them. For example, theTest Spycould save those values in a file or in a Registry [PEAA] object.
Motivating Example
The following test verifies the basic functionality of removing a flight but does not verify the indirect outputs of the SUT—namely, the fact that the SUT is expected to log each time a flight is removed along with the date/time and username of the requester.
public void testRemoveFlight() throws Exception { // setup FlightDto expectedFlightDto = createARegisteredFlight(); FlightManagementFacade facade = new FlightManagementFacadeImpl(); // exercise facade.removeFlight(expectedFlightDto.getFlightNumber()); // verify assertFalse("flight should not exist after being removed", facade.flightExists( expectedFlightDto. getFlightNumber())); }
Refactoring Notes
We can add verification of indirect outputs to existing tests using a Replace Dependency with Test Double refactoring. It involves adding code to the fixture setup logic of the tests to create theTest Spy, configuring theTest Spywith any values it needs to return, and installing it. At the end of the test, we add assertions comparing the expected method names and arguments of the indirect outputs with the actual values retrieved from theTest Spyusing theRetrieval Interface．
Example: Test Spy
In this improved version of the test, logSpy is ourTest Spy．The statement facade.setAuditLog(logSpy) installs theTest Spyusing theSetter Injectionpattern (seeDependency Injection). The methods getDate, getActionCode, and so on are theRetrieval Interfaceused to access the actual arguments of the call to the logger.
公共空间testRemoveFlightLogging_recordingTestStub() throws Exception { // fixture setup FlightDto expectedFlightDto = createAnUnregFlight(); FlightManagementFacade facade = new FlightManagementFacadeImpl(); // Test Double setup AuditLogSpy logSpy = new AuditLogSpy(); facade.setAuditLog(logSpy); // exercise facade.removeFlight(expectedFlightDto.getFlightNumber()); // verify assertFalse("flight still exists after being removed", facade.flightExists( expectedFlightDto. getFlightNumber())); assertEquals("number of calls", 1, logSpy.getNumberOfCalls()); assertEquals("action code", Helper.REMOVE_FLIGHT_ACTION_CODE, logSpy.getActionCode()); assertEquals("date", helper.getTodaysDateWithoutTime(), logSpy.getDate()); assertEquals("user", Helper.TEST_USER_NAME, logSpy.getUser()); assertEquals("detail", expectedFlightDto.getFlightNumber(), logSpy.getDetail()); }
这个测试取决于以下的定义theTest Spy:
public class AuditLogSpy implements AuditLog { // Fields into which we record actual usage information private Date date; private String user; private String actionCode; private Object detail; private int numberOfCalls = 0; // Recording implementation of real AuditLog interface public void logMessage(Date date, String user, String actionCode, Object detail) { this.date = date; this.user = user; this.actionCode = actionCode; this.detail = detail; numberOfCalls++; } // Retrieval Interface public int getNumberOfCalls() { return numberOfCalls; } public Date getDate() { return date; } public String getUser() { return user; } public String getActionCode() { return actionCode; } public Object getDetail() { return detail; } }
Of course, we could have implemented theRetrieval Interfaceby making the various fields of our spy public and thereby avoided the need for accessor methods. Please refer to the examples inHard-Coded Test Doublefor other implementation options.