CS696: Doc 9, Marshalling & Serialization

	CS 696 Emerging Technologies: Distributed Objects

Spring Semester, 1998
Marshalling & Serialization
To Lecture Notes Index
© 1998, All Rights Reserved, SDSU & Roger Whitney
San Diego State University -- This page last updated 21-Apr-98

Contents of Doc 9, Marshalling & Serialization

Reference

Java in a NutShell 2nd Ed, Flanagan, chapter 9

Java 1.1 API

Doc 9, Marshalling & Serialization Slide # 2

Marshaling is process of encoding object to put them on the wire (network)

Unmarshaling is the process of decoding from the wire and placing object in the address space

RMI uses serialization (deserialization) to perform marshaling (Unmarshaling)

Doc 9, Marshalling & Serialization Slide # 3

Serialization

Serialization allows objects to be converted to a sequence of bytes

The sequence of bytes can be stored in a file, database, sent to a remote machine etc.

The sequence of bytes can be used to recreate the original object

Serializing

Creating the sequence of bytes from an object

Deserializing

Recreating the object from the above generated bytes

Actually creates a new object with the fields containing the same values as the original object

If the fields are also serializable object then they are also "recreated"

Doc 9, Marshalling & Serialization Slide # 4

Making an Object Serializable

The object's class must implement the interface

java.io.Serializable

Serializable has no methods
Example

import java.io.Serializable;

class Roger implements Serializable
   {
   public int lowBid;
   
   public Roger(int lowBid )
      {
      this.lowBid = lowBid;
      }
   
   public String toString()
      {
      return " " + lowBid;
      }
   }
Doc 9, Marshalling & Serialization Slide # 5

Making Object Valid Remote Parameters

The Parameter Object

package whitney.rmi.examples.basic;

public class SerializedFoo implements java.io.Serializable
   {
   String name = "Roger";
   }

The Remote Interface

package whitney.rmi.examples.basic;

public interface SerializedParameter extends java.rmi.Remote 
   {
   public SerializedParameter tryMe( SerializedFoo input ) 
      throws java.rmi.RemoteException;
   }

Doc 9, Marshalling & Serialization Slide # 6

The Server

package whitney.rmi.examples.basic;

import java.rmi.*;
import java.rmi.server.UnicastRemoteObject;
import java.rmi.registry.Registry;
import sdsu.rmi.registry.Registrar;
import sdsu.util.ProgramProperties;
import java.io.IOException;

public class SerializedParameterServer 
      extends UnicastRemoteObject
      implements SerializedParameter
   {
   public SerializedParameterServer() throws 
      RemoteException {}

   public SerializedParameter tryMe( SerializedFoo input ) 
      throws RemoteException 
      {
      return this;
      }

   public static String RMI_NAME = "cs696/SerializedParameter";
   static String serverClass =
       "whitney.rmi.examples.basic.SerializedParameterServer";
   
   public static void main(String args[]) throws RemoteException, IOException
      {
      int port = getPort( args );

      String nameList = Registrar.verboseRebind( port, RMI_NAME, serverClass);
      System.out.println( nameList );
      }
      
   private static int getPort( String args[] ) throws IOException
      {
      ProgramProperties flags = new ProgramProperties( args );
      int defaultPort = Registry.REGISTRY_PORT;
      int port = flags.getInt( "p", defaultPort );

      return port; 
      }
   }

Doc 9, Marshalling & Serialization Slide # 7

The Client

package whitney.rmi.examples.basic;

import java.rmi.*;
import java.rmi.registry.Registry;
import java.net.MalformedURLException;
import java.io.IOException;
import sdsu.util.ProgramProperties;

public class SerializedParameterClient 
   {
   public static void main(String args[]) 
      {
      try 
         {
         String serverLabel = 
            SerializedParameterServer.RMI_NAME;
         String server = getServerURL( args, serverLabel );
         
         SerializedParameter remote = 
            (SerializedParameter) Naming.lookup( server );
            
         remote.tryMe( new SerializedFoo() );
            } 
      catch ( Exception error) 
         {
         error.printStackTrace();
         }
      }
   
   private static String getServerURL( String args[], String serverLabel ) throws IOException
      {
      String hostKey = "h";
      String portKey = "p";
      
      ProgramProperties flags = new ProgramProperties( args );
      
      String host = null;
      if ( flags.containsKey( hostKey ) )
         host = flags.getString( hostKey );
      else
         {
         System.out.println( "Missing flag " + hostKey );
         System.exit( 0 );
         }
      
      int defaultPort = Registry.REGISTRY_PORT;
      int port = flags.getInt( portKey, defaultPort );
      
      return "rmi://" + host + ":" + port + "/" + serverLabel;
      }
   }

Doc 9, Marshalling & Serialization Slide # 8

Serializing and Deserializing Objects
The writeObject method of ObjectOutputStream serializes objects

The readObject method of ObjectInputStream deserializes objects
Example

import java.io.*;

class SerializeDeserialize {
   public static void main( String args[] ) throws IOException {
      serialize();
      deserialize();
   }
      
   public static void serialize() throws Exception {
      OutputStream outputFile = 
            new FileOutputStream( "Serialized" );
      ObjectOutputStream cout = 
            new ObjectOutputStream( outputFile );
      cout.writeObject( new Roger( 1));
      cout.close();
   }   
   
   public static void deserialize() throws Exception {
      InputStream inputFile = 
         new FileInputStream( "Serialized" );
      ObjectInputStream cin = 
         new ObjectInputStream( inputFile );
      System.out.println( cin.readObject() );
   }
}

Output 1

Doc 9, Marshalling & Serialization Slide # 9

ObjectOutputStream Methods annotateClass(Class) close() defaultWriteObject() drain() enableReplaceObject(boolean) flush() replaceObject(Object) reset() write(byte[]) write(byte[], int, int) write(int) writeBoolean(boolean)

writeByte(int)

writeBytes(String)

writeChar(int)

writeChars(String)

writeDouble(double)

writeFloat(float)

writeInt(int)

writeLong(long)

writeObject(Object)

writeShort(int)

writeStreamHeader()

writeUTF(String)

public final void writeObject(Object obj) throws IOException

Throws: InvalidClassException

Something is wrong with a class used by serialization.

Throws: NotSerializableException

Some object to be serialized does not implement the java.io.Serializable interface.

Throws: IOException

Any exception thrown by the underlying OutputStream.

Doc 9, Marshalling & Serialization Slide # 10

ObjectInputStream Methods available() readInt()
close() readLine()
defaultReadObject() readLong()
enableResolveObject(boolean) readObject()
read() readShort()
read(byte[], int, int) readStreamHeader()
readBoolean() readUnsignedByte()
readByte() readUnsignedShort()
readChar() readUTF()
readDouble() registerValidation(ObjectInputValidation, int)
readFloat() resolveClass(ObjectStreamClass)
readFully(byte[]) resolveObject(Object)
readFully(byte[], int, int) skipBytes(int)

public final Object readObject() throws 
   OptionalDataException, 
   ClassNotFoundException, 
   IOException

Throws: ClassNotFoundException

Class of a serialized object cannot be found.

Throws: InvalidClassException

Something is wrong with a class used by serialization.

Throws: StreamCorruptedException

Control information in the stream is inconsistent.

Throws: OptionalDataException

Primitive data was found in the stream instead of objects.

Throws: IOException

Any of the usual Input/Output related exceptions.

Doc 9, Marshalling & Serialization Slide # 11

Serializable objects can contain other objects

import java.io.Serializable;
import java.util.Vector;

class Roger implements Serializable
   {
   public int lowBid;
   public int[] allBids = new int[10];
   public Vector bidderNames = new Vector();
   
   public Roger(int lowBid )
      {
      this.lowBid = lowBid;
      bidderNames.addElement( "Bill");
      }
   
   public String toString()
      {
      return " " + lowBid + " " + bidderNames;
      }
   }
Doc 9, Marshalling & Serialization Slide # 12

Multiple Items on One Stream
If more than one object of the same type is put on the stream, the header information for the class is sent only once.

import java.io.*;

class MultipleItems
   {
   public static void main( String args[] ) throws Exception
      {
      serialize();
      deserialize();
      }
      
   public static void serialize() throws IOException
      {
      OutputStream outputFile = 
         new FileOutputStream( "Serialized" );
      ObjectOutputStream cout = 
         new ObjectOutputStream( outputFile );

      cout.writeObject( new Roger( 2 ));

      cout.writeUTF( "Hi Mom" );
      cout.writeObject( "Hi Dad" );

      cout.writeFloat( 2.345F );
      cout.writeObject( new Roger( 3 ));

      cout.close();
      }
Doc 9, Marshalling & Serialization Slide # 13

//Multiple Items - Continued

   public static void deserialize() throws Exception
      {
      InputStream inputFile = 
         new FileInputStream( "Serialized" );
      ObjectInputStream cin = 
            new ObjectInputStream( inputFile );

      System.out.println( cin.readObject() );

      System.out.println( cin.readUTF() );
      System.out.println( cin.readObject() );

      System.out.println( cin.readFloat() );
      System.out.println( cin.readObject() );
      }
   }

Output 2 [Bill]
Hi Mom
Hi Dad
2.345
3 [Bill]

Doc 9, Marshalling & Serialization Slide # 14

Saving and Recovering - A Simple Example

import java.io.*;
import java.util.*;

// Cheap demo class for something to save
public class Student implements Serializable
   {
   String name;
   String address;
   
   public Student( String name, String address )
      {
      this.name = name;
      this.address = address;
      }
   
   public String toString()
      {
      return name + "@" + address + "\n";
      }
   }

Doc 9, Marshalling & Serialization Slide # 15

//Saving and Recovering - A Simple Example


public class StudentList implements Serializable
   {
   Vector list = new Vector();
   String storageFileName;
   
   public StudentList( String fileForStorage)
      {
      storageFileName = fileForStorage;
      }
   
   public void addStudent( Student addToList )
      {
      list.addElement( addToList );
      }
      
   public String toString()
      {
      return list.toString();
      }
   
   // Have the list save itself in a file
   public void save() throws IOException
      {
      OutputStream outputFile = 
         new FileOutputStream( storageFileName );
      ObjectOutputStream cout = 
         new ObjectOutputStream( outputFile );
      cout.writeObject( this );
      cout.close();
      }
   
Doc 9, Marshalling & Serialization Slide # 16

//Saving and Recovering - A Simple Example //StudentList continued

   // Recover a StudentList from a file
   public static StudentList fromFile( String studentListFile) 
      throws 
         OptionalDataException, 
         ClassNotFoundException, 
         IOException

      {
      InputStream inputFile = 
         new FileInputStream( studentListFile );
      ObjectInputStream cin = 
         new ObjectInputStream( inputFile );

      StudentList recoveredList = 
         (StudentList) cin.readObject();

      cin.close();
      return recoveredList;
      }
   }

Doc 9, Marshalling & Serialization Slide # 17

//Saving and Recovering - A Simple Example The driver program and Sample output

class Test
   public static void main( String args) throws Exception
      {
      StudentList cs535 = new StudentList( "cs535File");
      cs535.addStudent( 
         new Student( "Roger", "whitney@rohan"));

      cs535.addStudent( 
         new Student( "Sam", "masc1232@rohan"));
      cs535.addStudent( 
         new Student( "Ngu", "masc1111@rohan"));
      cs535.addStudent( 
         new Student( "catMan", "masc43221@rohan"));

      System.out.println( cs535);

      cs535.save();
      
      StudentList recoveredClass = 
            StudentList.fromFile( "cs535File" );

      System.out.println( recoveredClass);
      }   
   }

Output [Roger@whitney@rohan
, Sam@masc1232@rohan
, Ngu@masc1111@rohan
, catMan@masc43221@rohan
]
[Roger@whitney@rohan
, Sam@masc1232@rohan
, Ngu@masc1111@rohan
, catMan@masc43221@rohan
]

Doc 9, Marshalling & Serialization Slide # 18

Non-Serializable Objects
Some objects should not be serialized

objects that hold system resources

Some values or objects can be recomputed or are not important to serialize

Non-Serializable Fields - transient

If an object has a field that should not be serialized declare the field transient

That field will not be included in the serialization of the object

Doc 9, Marshalling & Serialization Slide # 19

transient Example


import java.io.Serializable;
import java.util.Vector;

class Roger implements Serializable
   {
   private int lowBid;
   private transient float averageBid;
   private int highBid;
   
   public Roger(int lowBid, int highBid )
      {
      this.lowBid = lowBid;
      this.highBid = highBid;
      averageBid = (lowBid + highBid) /2;
      }
   
   public String toString()
      {
      return " " + lowBid + " " + averageBid;
      }
   }

Doc 9, Marshalling & Serialization Slide # 20

// transient Example - Continued

class TransientExample
   {
   public static void main( String args[] ) throws Exception
      {
      serialize();
      deserialize();
      }
      
   public static void serialize() throws IOException
      {
      OutputStream outputFile = 
         new FileOutputStream( "Serialized" );
      ObjectOutputStream cout = 
         new ObjectOutputStream( outputFile );

      cout.writeObject( new Roger( 1, 5 ));

      cout.close();
      }   
   public static void deserialize() throws Exception
      {
      InputStream inputFile = 
         new FileInputStream( "Serialized" );
      ObjectInputStream cin = 
         new ObjectInputStream( inputFile );

      System.out.println( cin.readObject() );
      }   
   }

Output 1 0.0

Doc 9, Marshalling & Serialization Slide # 21

Customizing Deserialization

The readObject() method allows you to customize the deserialization of an object

The readObject() must have the signature given below

readObject must be implemented in the class of the object to be deserialized

   private void readObject( ObjectInputStream in)
      {
      // this is normally called first
      in.defaultReadObject();
      
      //Now you do the custom work
      }

defaultReadObject() throws the following exceptions

Throws: ClassNotFoundException

if the class of a serialized object could not be found.

Throws: IOException

if an I/O error occurs.

Throws: NotActiveException (subclass of IOException)

if the stream is not currently reading objects.

Doc 9, Marshalling & Serialization Slide # 22

Example of readObject

import java.io.Serializable;
import java.io.ObjectInputStream;
import java.io.IOException;

class Roger implements Serializable
   {
   private int lowBid;
   private transient float averageBid;
   private int highBid;
   
   public Roger(int lowBid, int highBid )
      {
      this.lowBid = lowBid;
      this.highBid = highBid;
      averageBid = (lowBid + highBid)/2;
      }
   
   private void readObject( ObjectInputStream in) throws 
         IOException, ClassNotFoundException
      {
      in.defaultReadObject();
      
      averageBid = (lowBid + highBid)/2;
      }
      
   public String toString()
      {
      return " " + lowBid + " " + averageBid;
      }
   }
Doc 9, Marshalling & Serialization Slide # 23

// Example of readObject - Continued

class ReadObjectExample
   {
   public static void main( String args[] ) throws Exception
      {
      serialize();
      deserialize();
      }
      
   public static void serialize() throws IOException
      {
      OutputStream outputFile = 
         new FileOutputStream( "Serialized" );
      ObjectOutputStream cout = 
         new ObjectOutputStream( outputFile );

      cout.writeObject( new Roger( 1, 5 ));

      cout.close();
      }   
   public static void deserialize() throws Exception
      {
      InputStream inputFile = 
         new FileInputStream( "Serialized" );
      ObjectInputStream cin = 
         new ObjectInputStream( inputFile );

      System.out.println( cin.readObject() );
      }   
   }

Output

 1 3.0
Doc 9, Marshalling & Serialization Slide # 24

Customizing Serialization

The writeObject() method allows you to customize the serialization of an object

The writeObject() must have the signature given below

writeObject must be implemented in the class of the object to be deserialized

   private void writeObject( ObjectOutputStream out )          {
      // normally do the custom work before
      // you call defaultWriteObject()

      out.defaultWriteObject();
      }

defaultWriteObject() throws the following exceptions

Throws: IOException

if an I/O error occurs.

Throws: NotActiveException (subclass of IOException)

if the stream is not currently writing objects.

Doc 9, Marshalling & Serialization Slide # 25

Class Versions

The Problem
Start with a class:

class Unstable
   {
   double hereToday;
   }

Serialize an Unstable object to a file for later use

Now change the class

class Unstable
   {
   char goneTomorrow;
   }

After the class has changed, try to deserialize the object!

What should happen?
What could happen? (Hint nothing very good)

Doc 9, Marshalling & Serialization Slide # 26

Class serialVersionUID
Java generates a serialVersionUID for each class

When deserializing an object, the serialVersionUID of the deserialized object and it class are checked

If they do not agree, an exception is thrown
Information Used to Compute the serialVersionUID
1. Class name
2. The class modifiers
3. The name of each interface
4. For each field of the class(except private static and private transient fields):

The name of the field
The modifiers of the field
The descriptor of the field

5. For each method including constructors, except private methods and constructors:

The name of the method
The modifiers of the method
The descriptor of the method

Doc 9, Marshalling & Serialization Slide # 27

Controlling The Class Version
By declaring in your class a field of type and name

static final long serialVersionUID

you override the default method of computing the classes version number.

This is done when changes are made to a class that don't break deserialization of older objects.

For example start with the class:

class Sam implements Serializable
   {
   private int lowBid;
   
   public Sam(int lowBid)
      {
      this.lowBid = lowBid;
      }
   }

Now add a toString method to the class. This will change the class version. Objects serialized with the original Sam class will not be serializable with the Sam class containing the toString(). But clearly we have not changed the class in any way to make it impossible to deserialize the old objects. If we added a serialVersionUID to the class and did not change its value, then deserialization would work with the modified class and old objects!

Doc 9, Marshalling & Serialization Slide # 28

Example of serialVersionUID
Start with this class:

class Roger implements Serializable
   {
   static final long serialVersionUID = 1L;
   private int lowBid;
   
   public Roger(int lowBid )
      {
      this.lowBid = lowBid;
      }
   }

Serialize it with the following program. The rest of the example will read this serialized version of the Roger object

class SerializeRoger
   {
   public static void main( String args[] ) throws Exception
      {
      OutputStream outputFile = 
         new FileOutputStream( "Serialized" );
      ObjectOutputStream cout = 
         new ObjectOutputStream( outputFile );

      cout.writeObject( new Roger( 1, 5 ));

      cout.close();
      }
   }

Doc 9, Marshalling & Serialization Slide # 29

Example of serialVersionUID - Continued

Change the class as follows and recompile

class Roger implements Serializable
   {
   static final long serialVersionUID = 1L;
   private int lowBid;
   
   public Roger(int lowBid )
      {
      this.lowBid = lowBid;
      }

   public String toString()
      {
      return " " + lowBid + " " + averageBid;
      }
   }

Now deserialize the original Roger object with the program. There is no problem as the class has not changed any data

class ReadRogerObejct
   {
   public static void main( String args[] ) throws Exception
      {
      InputStream inputFile = 
         new FileInputStream( "Serialized" );
      ObjectInputStream cin = 
         new ObjectInputStream( inputFile );

      System.out.println( cin.readObject() );
      }   
   }
Doc 9, Marshalling & Serialization Slide # 30

Example of serialVersionUID - Continued

Now we add some new fields to the class. Since they are not in the saved Roger object, we add the readObject method to make some reasonable values for the new fields. Compiling the class and deserialize in the original Roger object with the ReadRogerObejct class. It will work.


class Roger implements Serializable
   {
   static final long serialVersionUID = 1L;
   private int lowBid;
   private transient float averageBid;
   private int highBid;

   public Roger(int lowBid, int highBid )
      {
      this.lowBid = lowBid;
      this.highBid = highBid;
      averageBid = (lowBid + highBid)/2;
      }

   private void readObject( ObjectInputStream in) throws 
         IOException, ClassNotFoundException
      {
      in.defaultReadObject();
      if ( highBid == 0)
         highBid = lowBid;
      averageBid = (lowBid + highBid)/2;
      }

   public String toString()
      {
      return " " + lowBid + " " + averageBid;
      }
   }
Doc 9, Marshalling & Serialization Slide # 31

Example of serialVersionUID - Continued

We can change the class to the following and it will still work to deserialize the original Roger object. Note that the field "lowBid" is not in this class.


class Roger implements Serializable
   {
   static final long serialVersionUID = 1L;
   private int highBid;
   
   public Roger(int highBid)
      {
      this.highBid= highBid;
      }
   }

Doc 9, Marshalling & Serialization Slide # 32

Example of serialVersionUID - Continued

¿So what does not work?

This is actually two questions:

When will the deserialization throw an exception due to changes in the class?

What changes in the class logical require a new version?

Changing the type of a field causes the deserialization to fail. The following class will cause to the deserialization of the original Roger object to fail

class Roger implements Serializable
   {
   static final long serialVersionUID = 1L;
   private float lowBid;
   
   public Roger(int lowBid)
      {
      this.lowBid = lowBid;
      }
   }

The programmer will have to decide the answer to the second question on an individual basis.

Doc 9, Marshalling & Serialization Slide # 33

Serialver

When the class needs another version number use serialver to generate a new one

Sun recommends using the program serialver to generate the serialVersionUID for your class

This recommendation is given in the context of Java's RMI for distributing objects over a network

This recommendation is made for security reasons
Using serialvar
serialvar is located in the same directory as javac

1. Compile your class, make sure that the variable:

static final long serialVersionUID

is not defined in the class

2. run serialvar using the command line:

serialvar  classname

3. The output of serialvar is line of text to declare the variable serialVersionUID

Doc 9, Marshalling & Serialization Slide # 34

Example of Using serialvar

import java.io.Serializable;

class Roger implements Serializable
   {
   public int lowBid;
   
   public Roger(int lowBid )
      {
      this.lowBid = lowBid;
      }
   
   public String toString()
      {
      return " " + lowBid;
      }
   }

Put the above code in the file Roger.java

eli-> javac Roger.java
eli-> serialver Roger
Roger:    static final long serialVersionUID = -8462350894591099987L;
Doc 9, Marshalling & Serialization Slide # 35

RMI and serialvar

The first time an object is passed by copy, serialvar must run on the object's class, which takes time

If you do it manually before hand, you improve RMI's performance

visitors since 10-Feb-98