Client-Server Programming Spring Semester, 2005 Testing
	Previous     Lecture Notes Index     Next      © 2005, All Rights Reserved, SDSU & Roger Whitney San Diego State University -- This page last updated 25-Jan-05

CS 580 Client-Server Programming Spring Semester, 2005 Doc 2 Testing

Contents

Testing

Johnson's Law

Why Unit Testing

When to Write Unit Tests

What to Test

SUnit & JUnit

How to Use SUnit

TestCase methods of interest

Using JUnit

Running JUnit Using Eclipse

assert Methods

Testing the Tests

Test Fixtures

Suites – Multiple Test Classes

How to Test Exceptions

Testing and Hidden Methods/State

How to Test Hidden Methods Directly?

Method 1: Relax the protection level

Method 2: Use inner classes

Method 3: Use reflection

Running JUnit without Eclipse

Copyright ©, All rights reserved. 2005 SDSU & Roger Whitney, 5500 Campanile Drive, San Diego, CA 92182-7700 USA. OpenContent ( http://www.opencontent.org/opl.shtml) license defines the copyright on this document

.

References

JUnit Cookbook http://junit.sourceforge.net/doc/cookbook/cookbook.htm

JUnit Test Infected: Programmers Love Writing Tests http://junit.sourceforge.net/doc/testinfected/testing.htm

JUnit Javadoc: http://www.eli.sdsu.edu/java-SDSU/docs/

Brian Marick’s Testing Web Site: http://www.testing.com/

Testing for Programmers, Brian Marick, Available at: http://www.testing.com/writings.html

Reading Assignment

Test Infected - Programmers Love Writing Tests, http://junit.sourceforge.net/doc/testinfected/testing.htm

JUnit FAQ, http://junit.sourceforge.net/doc/faq/faq.htm

Programming with Assertions, http://java.sun.com/j2se/1.4/docs/guide/lang/assert.html

Testing

Johnson's Law

If it is not tested it does not work

Types of tests

• Unit Tests

Tests individual code segments

• Functional Tests

Test functionality of an application

Why Unit Testing

If it is not tested it does not work

The more time between coding and testing

• More effort is needed to write tests
• More effort is needed to find bugs
• Fewer bugs are found
• Time is wasted working with buggy code
• Development time increases
• Quality decreases

Without unit tests

• Code integration is a nightmare
• Changing code is a nightmare

• CS 580 Spring 05

  Doc 2, Testing Slide # 5

Why Automated Tests?

What wrong with:

• Using print statements
• Writing driver program in main
• Writing small sample programs to run code
• Running program and testing it be using it

Repeatability & Scalability

Need testing methods that:

• Work with N programmers working for K months (years)

• Help when modify code 6 months after it was written

• Check impact of code changes in rest of system

Practices that work in a school project may not be usable in industry

Standard industry practices may seem overkill in a school project

Work on building good habits and skills

We have a QA Team, so why should I write tests?

How long does it take QA to test your code?

How much time does your team spend working around bugs before QA tests?

How easy is it to find & correct the errors after QA finds them?

Most programmers have an informal testing process

With a little more work you can develop a useful test suite

When to Write Unit Tests

First write the tests

Then write the code to be tested

Writing tests first saves time

• Makes you clear of the interface & functionality of the code

• Removes temptation to skip tests

What to Test

Everything that could possibly break

Test values

Inside valid range Outside valid range On the boundary between valid/invalid

GUIs are very hard to test

Keep GUI layer very thin Unit test program behind the GUI, not the GUI

Common Things that Programs Handle Incorrectly

Adapted with permission from “A Short Catalog of Test Ideas” by Brian Marick, http://www.testing.com/writings.html

Strings

Test using empty String

Collections

Test using:

• Empty Collection
• Collection with one element
• Collection with duplicate elements
• Collections with maximum possible size

Numbers

Test using:

• Zero
• The smallest number
• Just below the smallest number
• The largest number
• Just above the largest number

SUnit & JUnit

Free frameworks for Unit testing

SUnit originally written by Kent Beck 1994

• Built into VisualWorks 7.x

JUnit written by Kent Beck & Erich Gamma

• Available at: http://www.junit.org/

Ports are available in

See http://www.xprogramming.com/software.htm to download ports

How to Use SUnit

1. Make test class a subclass of TestCase

2. Make test methods

The framework treats methods starting with 'test' as test methods

3. Run the tests

• You can run the test using TestRunner

      TestRunner open

• Use Browser SUnit Extensions
• Load parcel/RBSUnitExtensions.pcl to run the tests from the browser

• CS 580 Spring 05

  Doc 2, Testing Slide # 13

Broswer SUnit Extensions

Sample TestClass

Smalltalk.CS535 defineClass: #TestCounter
   superclass: #{XProgramming.SUnit.TestCase}
   indexedType: #none
   private: false
   instanceVariableNames: 'counter '
   classInstanceVariableNames: ''
   imports: ''
   category: 'Course-Examples'

CS535.TestCounter methodsFor: 'testing'

setUp
   counter := Counter new.
   
tearDown
   counter := nil.
   
testDecrease
   counter decrease.
   self assert: counter count = -1.
   
testDecreaseWithShould
   "Just an example to show should: syntax"
   counter decrease.
   self should: [counter count = -1].
   
testIncrease
   self deny: counter isNil.
   counter increase.
   self assert: counter count = 1.
   
testZeroDivide
   "Just an example to show should:raise: syntax"
   self 
      should: [1/0]
      raise: ZeroDivide.
   
   self 
      shouldnt: [1/2]
      raise: ZeroDivide

TestCase methods of interest

Methods to assert conditions:

   assert: aBooleanExpression
   assert: aBooleanExpression description: aString
   deny: aBooleanExpression
   deny: aBooleanExpression description: aString
   
   should: [aBooleanExpression]
   should: [aBooleanExpression] raise: AnExceptionClass
   
   shouldnt: [aBooleanExpression]
   shouldnt: [aBooleanExpression] raise: AnExceptionClass
   
   signalFailure: aString

setUp

Called before running each test method in the class.

Used to:

Open files Open database connections Create objects needed for test methods

tearDown

Called after running each test method in the class.

Used to:

Close files Close database connections Nil out references to objects

CS 580 Spring 05
Doc 2, Testing Slide # 16

Using JUnit

Example

Goal: Implement a Stack containing integers.

Tests:

• Subclass junit.framework.TestCase

• Methods starting with 'test" are run by TestRunner

First tests for the constructors:

public class  TestStack extends TestCase {
   
   
   public void testDefaultConstructor() {
      Stack test = new Stack();
      assertTrue("Default constructor", test.isEmpty() );
   }
      
   public void testSizeConstructor() {
      Stack test = new Stack(5);
      assertTrue( test.isEmpty() );
   }
}

First part of the Stack

package example;
   
public class Stack  {
   int[] elements;
   int topElement = -1;
   
   public Stack() {
    this(10); 
   }
   
   public Stack(int size) { 
      elements = new int[size]; 
   }
   
   public boolean isEmpty() {
      return topElement == -1;
   }
}
CS 580 Spring 05
Doc 2, Testing Slide # 18

Running JUnit Using Eclipse

Select JUnit Test Case

There seems to be at least three different menus one can use to create a test case in eclipse. In the Java perspective after creating the class you create test case class by selecting the “JUnit Test Case” item in the “New” submenu of the “File” menu. Or you can select the “JUnit Test Case” item in the “New Class” popup menu obtained by clicking on the new class icon (see below)

When you ask eclipse to create a test case you get dialog window like:

Run the Test

The default settings in the above dialog are good enough for this example. Click on the finish button at the bottom of the window. Add the two methods to the test class. Then run the class as a JUnit test. You do this by in the “Run” menu select the “Run as...” item and then selecting the “JUnit Test” item in the submenu. After the test case runs you will see a new panel that displays the results of the run.

assert Methods

assertTrue()
assertFalse()
assertEquals()
assertNotEquals()
assertSame()
assertNotSame()
assertNull()
assertNotNull()
fail()

For a complete list of the assert methods & arguments see

http://www.junit.org/junit/javadoc/3.8/index.htm/allclasses-frame.html/junit/junit/framework/Assert.html/Assert.html

JUnit, Java 1.4 & assert

JUnit had a method called assert()

Java 1.4 makes assert a reserved word

JUnit starting 3.7 replaces assert() with assertTrue()

Use JUnit 3.7 or later with JDK 1.4

To use JDK 1.4 asserts:

• Compile with option -source 1.4

   java -source 1.4 programFile.java

• Run with the option -ea

   java -ea programFile

Testing the Tests

If can be useful to modify the code to break the tests

package example;
   
public class Stack  {
   int[] elements;
   int topElement = -1;
   
   etc.
   
   public boolean isEmpty() {
      return topElement == 1;
   }
}

Why Test the Tests?

One company had an automatic build and test cycle that ran at night. The daily build was created and all the tests were run at night. The test results were available first thing in the morning. One night the build process crashed, so the daily build was not made. Hence there was no code to test. Still 70% of the tests passed. If they had tested their tests, they would have discovered immediately that their tests were broken.

Test Fixtures

Before each test setUp() is run

After each test tearDown() is run

package example;
   
import junit.framework.TestCase;
   
public class  StackTest extends TestCase {
   Stack test;
      
   public void setUp() {
      test = new Stack(5);
      for (int k = 1; k <=5;k++)
         test.push( k);   
   }
   
   public void testPushPop() {
   
      for (int k = 5; k >= 1; k--)
         assertEquals( "Pop fail on element " + k,  test.pop() , k);
   }
}

Suites – Multiple Test Classes

Multiple test classes can be run at the same time

Add Queue & TestQueue to Stack classes

package example;
   
import junit.framework.TestCase;
   
public class TestQueue extends TestCase{
   
   public void testConstructor() {
      Queue test = new Queue();
      assertTrue( test.isEmpty());
   }
}

package example;
   
import java.util.Vector;
   
public class  Queue{
   Vector elements = new Vector();
   public boolean isEmpty() {
      return elements.isEmpty();
   }
}

Using a Suite to Run Multiple Test Classes

Running AllTests runs the tests in

• StackTest
• QueueTest

package example;
import junit.framework.Test;
import junit.framework.TestSuite;
   
public class AllTests  {
   static public void main(String[] args) {
       TestRunner.run( example.AllTests.suite());;
   }
   
   public static Test suite(){
      TestSuite suite = new TestSuite("Test for example");
      suite.addTestSuite(StackTest.class);
      suite.addTestSuite(QueueTest.class);
      return suite;
   }
}

Eclipse will generate the above class for you. In the file menu select “New”, then in the submenu select “other”. In the dialog window select “JUnit Test Suite” which is under Java and then under JUnit.

CS 580 Spring 05
Doc 2, Testing Slide # 27

How to Test Exceptions

At times you may wish to test input to methods that will cause an exception to be thrown

Here is an example of a test that

• passes when an exception is thrown
• fails when the exception is not thrown

Example is from the JUnit FAQ

public void testIndexOutOfBoundsException() {
      
    ArrayList list = new ArrayList(10);
      
    try {
   
        Object o = list.get(11);
   
        fail("Should raise an IndexOutOfBoundsException");
   
    } catch (IndexOutOfBoundsException success) {}
}

Testing and Hidden Methods/State

Issues:

• How does one test hidden methods?

• Direct access to an object's state can reduce the time needed to write a test

Testing Hidden Methods One Position Don't Do it

Pro:

• Can not test everything
• Clients of an object only care if public interface works correctly
• Testing public interface also tests hidden methods
• Hidden methods are more likely to change, requiring changes to the tests

The basic idea is to work smarter not harder. One cannot completely test each class, and one does not have infinite time two write tests one should write the most effective tests possible. Tests of the public interface of class will also test hidden methods of the class. Bugs in hidden methods that never affect the public methods are not a problem. Since the most important thing is that the pubic interface works correctly, concentrate your tests on the public interface.

Con:

• The closer the test is to the code it tests the easier the test
• Bugs in hidden methods can make it hard to debug public methods

My experience is that the more code I write without tests, the more time I spend on finding and correcting bugs. How many times have you spent hours (days?) tracking down a bug that turned out to a simple bug in some simple untested method, which would have been easy to test? The argument that one cannot test everything and must make effective use of ones testing time is correct. Given the differences in programmer skill level, programmer experience, etc. everyone has to work out their own solution to this. The XP solution is to try to test everything that could possible break. Since most students are not used to testing, you have to fight the habit of not testing and testing after you have completely finished coding. Given the current state of affairs in commercial software, the industry has a lot to learn about testing.

How to Test Hidden Methods Directly?

Method 1: Relax the protection level

In Java one can

• Make the method package level access
• Place the test class in the same package as the tested code

Pro:

• Makes it possible to test the hidden method
• Clients outside the package can not access the method

Con:

• Clients in the package may then use the method
• Requires organizational discipline to avoid using the method

You should comment the method to inform the clients that the method is not to be used

How to Test Hidden Methods Directly?

Method 2: Use inner classes

import junit.framework.TestCase;
   
public class Foo {
    private int value;
   
    private void bar() {
        value = 10;
    }
   
    public static class FooTest extends TestCase {
        public FooTest(String name) {
            super(name );
        }
   
        public void testBar() {
            Foo a = new Foo();
            a.bar();
            assert( 10 == a.value );
        }
    }
}

Pro:

• Provides access to all methods/fields
• Test does not have to be shipped with production code
• Test stays with tested class

Con:

• Source files are bit harder to read
• Must remove all inner $class files from production code
• Test not with other test classes

How to Test Hidden Methods Directly?

Method 3: Use reflection

Pro:

• Java reflection provides access to all methods/fields of a class
• Does not require any changes to tested class

Con:

• Reflection can be slow
• Reflection is cumbersome to use
• Requires setting permission files

See http://www.eli.sdsu.edu/courses/fall98/cs596/notes/reflection/reflection.html for more information about reflection

Running JUnit without Eclipse

JUnit has three interfaces

• Text (junit.textui.*)

Fastest to run

• AWT (junit.ui.*)

• Swing (junit.swingui.*)

Can reload class files so you can Run TestRunner once Recompile program until it passes tests

Starting Swingui TestRunner

Make sure your classpath includes the code to tested

On Rohan use:

java  junit.swingui.TestRunner
   

You get a window similar to that on the next page

Enter the full name of the test class

Click on the Run button

If there are errors/failures select one

You will see a stack trace of the error

The “...” button will search for all test classes in your classpath

Swing version of JUnit TestRunner

Running the textui TestRunner

Sample Program using main

public class Testing {
   public static void main (String args[]) {
      junit.textui.TestRunner.run( example.TestStack.class);
   }
}

Output

..
Time: 0.067
   
OK (2 tests)
   
java has exited with status 0.

Previous     visitors since 25-Jan-05     Next