CS 580 Client-Server Programming Fall Semester, 2002 Testing |
||
---|---|---|
© 2002, All Rights Reserved, SDSU & Roger Whitney San Diego State University -- This page last updated 29-Sep-02 |
Reading Assignment
Test Infected - Programmers Love Writing Tests, http://junit.sourceforge.net/doc/testinfected/testing.htm
JUnit FAQ, http://junit.sourceforge.net/doc/faq/faq.htm
Programming with Assertions, http://java.sun.com/j2se/1.4/docs/guide/lang/assert.html
Lecture Source Code
Java – CVS module testExamples
Smalltalk – Store package testExamples
Testing
Johnson's Law
If it is not tested it does not work
Types of tests
Why Unit Testing
If it is not tested it does not work
The more time between coding and testing
Repeatability & Scalability
Need testing methods that:
We have a QA Team, so why should I write tests?
How long does it take QA to test your code?
How much time does your team spend working around bugs before QA tests?
How easy is it to find & correct the errors after QA finds them?
Most programmers have an informal testing process
With a little more work you can develop a useful test suite
When to Write Unit Tests
First write the tests
Then write the code to be tested
Writing tests first saves time
SUnit & JUnit
Free frameworks for Unit testing
SUnit originally written by Kent Beck 1994
.NET |
Ada |
AppleScript |
C |
C# |
C++ |
Curl |
Delphi |
Eiffel |
Eiffel |
Flash |
Forte
4GL
|
Gemstone/S |
Haskell |
HTML |
Jade |
LISP |
Objective-C |
Oracle |
Palm |
Perl |
Php |
PowerBuilder |
Python |
Ruby |
Scheme |
Smalltalk |
Visual
Basic
|
XML |
XSLT |
|
|
How to Use SUnit
1. Make test class a subclass of TestCase
2. Make test methods
TestRunner open
Sample TestClass
Smalltalk.CS535 defineClass: #TestCounter superclass: #{XProgramming.SUnit.TestCase} indexedType: #none private: false instanceVariableNames: 'counter ' classInstanceVariableNames: '' imports: '' category: 'Course-Examples'
CS535.TestCounter methodsFor: 'testing'
setUp counter := Counter new.
tearDown counter := nil.
testDecrease counter decrease. self assert: counter count = -1.
testDecreaseWithShould "Just an example to show should: syntax" counter decrease. self should: [counter count = -1].
testIncrease self deny: counter isNil. counter increase. self assert: counter count = 1.
testZeroDivide "Just an example to show should:raise: syntax" self should: [1/0] raise: ZeroDivide. self shouldnt: [1/2] raise: ZeroDivide
TestCase methods of interest
Methods to assert conditions:
assert: aBooleanExpression deny: aBooleanExpression should: [aBooleanExpression] should: [aBooleanExpression] raise: AnExceptionClass shouldnt: [aBooleanExpression] shouldnt: [aBooleanExpression] raise: AnExceptionClass signalFailure: aString setUp
tearDown
Using JUnit
Example
Goal: Implement a Stack containing integers.
Tests:
public class TestStack extends TestCase { //required constructor public TestStack(String name) { super(name); } public void testDefaultConstructor() { Stack test = new Stack(); assertTrue("Default constructor", test.isEmpty() ); } public void testSizeConstructor() { Stack test = new Stack(5); assertTrue( test.isEmpty() ); } }
First part of the Stack
package example; public class Stack { int[] elements; int topElement = -1; public Stack() { this(10); } public Stack(int size) { elements = new int[size]; } public boolean isEmpty() { return topElement == -1; } }
Running JUnit
JUnit has three interfaces
Starting Swingui TestRunner
Make sure your classpath includes the code to tested
On Rohan use:
java junit.swingui.TestRunnerYou get a window similar to that on the next page
Enter the full name of the test class
Click on the Run button
If there are errors/failures select one
You will see a stack trace of the error
The “...” button will search for all test classes in your classpath
Swing version of JUnit TestRunner
Running the textui TestRunner
Sample Program using main
public class Testing { public static void main (String args[]) { junit.textui.TestRunner.run( example.TestStack.class); } }
Output
.. Time: 0.067 OK (2 tests) java has exited with status 0.
assert Methods
assertTrue()
assertFalse()
assertEquals()
assertNotEquals()
assertSame()
assertNotSame()
assertNull()
assertNotNull()
fail()
For a complete list of the assert methods & arguments see
http://www.junit.org/junit/javadoc/3.8/index.htm/allclasses-frame.html/junit/junit/framework/Assert.html/Assert.html
JUnit, Java 1.4 & assert
JUnit had a method called assert()
Java 1.4 makes assert a reserved word
JUnit starting 3.7 replaces assert() with assertTrue()
Use JUnit 3.7 or later with JDK 1.4
To use JDK 1.4 asserts:
java -source 1.4 programFile.java
java -ea programFile
Testing the Tests
If can be useful to modify the code to break the tests
package example; public class Stack { int[] elements; int topElement = -1; etc. public boolean isEmpty() { return topElement == 1; } }
Result of running Textui.TestRunner .F.F
Time: 0.113
There were 2 failures:
1) testDefaultConstructor(example.TestStack)junit.framework.AssertionFailedError: Default constructor
at example.TestStack.testDefaultConstructor(TestStack.java:22)
at Testing.main(Testing.java:14)
2) testSizeConstructor(example.TestStack)junit.framework.AssertionFailedError
at example.TestStack.testSizeConstructor(TestStack.java:27)
at Testing.main(Testing.java:14)
FAILURES!!!
Tests run: 2, Failures: 2, Errors: 0
java has exited with status 0.
Why Test the Tests?
One company had an automatic build and test cycle that ran at night. The daily build was created and all the tests were run at night. The test results were available first thing in the morning. One night the build process crashed, so the daily build was not made. Hence there was no code to test. Still 70% of the tests passed. If they had tested their tests, they would have discovered immediately that their tests were broken.
Test Fixtures
Before each test setUp() is run
After each test tearDown() is run
package example; import junit.framework.TestCase; public class StackTest extends TestCase { Stack test; public StackTest(String name) { super(name); } public void setUp() { test = new Stack(5); for (int k = 1; k <=5;k++) test.push( k); } public void testPushPop() { for (int k = 5; k >= 1; k--) assertEquals( "Pop fail on element " + k, test.pop() , k); } }
Suites – Multiple Test Classes
Multiple test classes can be run at the same time
Add Queue & TestQueue to Stack classes
package example; import junit.framework.TestCase; public class TestQueue extends TestCase{ public TestQueue ( String name){ super(name); } public void testConstructor() { Queue test = new Queue(); assert( test.isEmpty()); } }
package example; import java.util.Vector; public class Queue{ Vector elements = new Vector(); public boolean isEmpty() { return elements.isEmpty(); } }
Using a Suite to Run Multiple Test Classes
Running AllTests in TestRunner runs the test in
package example; import junit.framework.TestSuite; import junit.textui.TestRunner; public class AllTests { static public void main(String[] args) { TestRunner.run( example.AllTests.suite());; } static public TestSuite suite() { TestSuite suite= new TestSuite(); Try { suite.addTest(new TestSuite(StackTest.class)); suite.addTest(new TestSuite(QueueTest.class)); } catch (Exception e){ } return suite; } }
Using Main We can use main to run the test via textui.TestRunner
The command:
java example.AllTestswill run all the tests in StackTest & QueueTest
package example; import junit.framework.TestSuite; import junit.textui.TestRunner; public class AllTests { static public void main(String[] args) { TestRunner.run(AllTests.class); } static public TestSuite suite() { same as last page } }
How to Test Exceptions
At times you may wish to test input to methods that will cause an exception to be thrown
Here is an example of a test that
public void testIndexOutOfBoundsException() { ArrayList list = new ArrayList(10); try { Object o = list.get(11); fail("Should raise an IndexOutOfBoundsException"); } catch (IndexOutOfBoundsException success) {} }
Testing and Hidden Methods/State
Issues:
Testing Hidden Methods One Position Don't Do it
Pro:
How to Test Hidden Methods Directly?
Method 1: Relax the protection level
In Java one can
How to Test Hidden Methods Directly? Method 2: Use inner classes
import junit.framework.TestCase; public class Foo { private int value; private void bar() { value = 10; } public static class FooTest extends TestCase { public FooTest(String name) { super(name ); } public void testBar() { Foo a = new Foo(); a.bar(); assert( 10 == a.value ); } } }
Pro:
How to Test Hidden Methods Directly? Method 3: Use reflection
Pro:
What to Test
Everything that could possibly break
Test values
Common Things that Programs Handle Incorrectly
Adapted with permission from “A Short Catalog of Test Ideas” by Brian Marick, http://www.testing.com/writings.html
Strings
Test using empty String
Collections
Test using:
Testing Network Code
Writing automated tests for network code can be hard
Make the network code very thin
Write automated tests for the non-network code
Example – SimpleDateServer
The SimpleDateServer has two methods
SimpleDateServer>>run | childSocket clientIOStream | [childSocket := serverSocket accept. clientIOStream := childSocket readAppendStream. clientIOStream lineEndTransparent. self processRequestOn: clientIOStream.] repeat
SimpleDateServer>>processRequestOn: anReadAppendStream | clientRequest | clientRequest := anReadAppendStream through: Character cr. (clientRequest startsWith: 'date') ifTrue: [anReadAppendStream nextPutAll: Time dateAndTimeNow printString; commit]. anReadAppendStream close
Issues in Writing test for processRequestOn:
processRequestOn: Input Need to create an ExternalAppendStream for input
Would be easier to test if arguments were:
Modified Server
SimpleDateServer>>run | childSocket | [childSocket := serverSocket accept. self processRequest: childSocket readStream lineEndTransparent response: childSocket writeStream lineEndTransparent] repeat
SimpleDateServer>>processRequest: aReadStream response: aWriteStream | clientRequest | clientRequest := aReadStream through: Character cr. (clientRequest startsWith: 'date') ifTrue: [aWriteStream nextPutAll: Date today printString; commit]. (clientRequest startsWith: 'time') ifTrue: [aWriteStream nextPutAll: Time now printString; commit]. aWriteStream close.
Some Tests
TestSimpleDateServer is subclass of TestCase
TestSimpleDateServer>>testDate | command response server serverDate serverDateString | command := 'date \' withCRs readStream. response := WriteStream on: String new. server := SimpleDateServer new. server processRequest: command response: response. serverDateString := response contents. serverDate :=Date readFrom: serverDateString readStream. self assert: serverDate = Date today.
TestSimpleDateServer>>testTime | command response server serverTimeString serverTime | command := 'time \' withCRs readStream. response := WriteStream on: String new. server := SimpleDateServer new. server processRequest: command response: response. serverTimeString := response contents. serverTime :=Time readFrom: serverTimeString readStream. self assert: (Time now asSeconds - serverTime asSeconds) < 2. TestSimpleDateServer>>testBadInput | command response server | command := 'fooBar \' withCRs readStream. response := WriteStream on: String new. server := SimpleDateServer new. server processRequest: command response: response. self assert: response contents trimBlanks isEmpty
Lessons
Keep the network layer thin
Server design may need to be modified to make it testable
See
Test networked code the easy way, Nelson Minar
http://www.javaworld.com/javaworld/jw-07-2002/jw-0719-networkunittest.html
for another example
Copyright ©, All rights reserved.
2002 SDSU & Roger Whitney, 5500 Campanile Drive, San Diego, CA 92182-7700 USA.
OpenContent license defines the copyright on this document.
Previous    visitors since 29-Sep-02    Next