Sunday, September 28, 2008

Going Native: Structures and Callbacks

This is a continuation of a simple overview of the Java Native Access (JNA) library. In the first part, I looked at the basics of mapping native library functions, with simple types.

JNA provides two key elements for the mapping Java to C libraries, namely structures and callbacks. Structures are automatically mapped from the native values and may be used as a type-safe pointer to a instance of the structure. Callbacks are provided via an extension of an interface, and the appropriate method is defined on the new interface.

Structures

Mapping native structs into Java classes is fairly simple via the Structure class. This class may be extended, and and any fields may be declared as public members of the new class.

For example, let's take the following C struct

struct {
char* name;
int count;
char* desc;
} record;

and map it into Java

public class Record extends Structure {
public String name;
public int count;
public String desc;
}


Once we have our structure defined, we can now create mappings for functions which operate on the record struct. In C:

int save(record* record);

and in our Java library interface:

public int save(Record record);


What if we want to pass this struct into a function to be populated? For example,

int load(record* record);

// and it's usage:
record r;
load(&r);
printf("record found: %s", r.name);

in which we load the record from a data store. We need a way, in Java, to pass this object to the native code, by reference. Fortunately, JNA provides a simple way of doing this.

public class Record extends Structure {
public class ByReference extends Record implements Structure.ByReference {}
...
}

// and it's usage:
Record.ByReference record = new Record.ByReference();
MyLibrary.INSTANCE.load(record);
System.out.println("record found: " + record.name);

This tag interface provides a very simple way of giving us this functionality.

Callbacks

Callbacks are provided by the Callback interface. By extending this interface we can add a method which will be used by the C library as the call back method.

Let say we have a method in our C library which will notify the application if a record has been deleted by another process. It provides us with a method to add a callback,

int register_delete_monitor( void* fn);

and the callback expected takes a parameter of a record.

In our Java library, we would create a Callback for this method:

interface DeletedRecordCallback extends Callback {
void callback (Record r);
}
...
// and in our Library Interface:
int register_delete_monitor (DeletedRecordCallback drc);

Now we can pass in an implementation of our DeletedRecordCallback and have it registered as a callback in the C library.

There are one caveat with the use of callbacks in JNA. The callback must not be garbage collected unless it's been unregistered. This is especially true for long lived callbacks. A call to a garbage-collected callback will crash the vm. This can be simply avoided by hanging on to the instance of the callback - a singleton will work quite well in this context. Otherwise, it is important to have the callback instance deregister itself in it's finalize method.

Overall Impressions

Coupling the power of JNA's type mappings, with its structure mapping and callback access, it is a very useful library for taking the difficulty out of mapping native code to java. It makes rapid development with existing native libraries much, much easier. Java-to-native mapping has never been easier.

Thursday, September 18, 2008

Going Native

I have a new project (and a new work) which involves starting and interacting with a python interpreter from the JVM.

I did my requisite Google search. At first, I thought I would do this using an existing library, JPype, but this works the other way around - the JVM is embedded in Python. So I moved to the next option, JNI. JNI is a pain in the ass, so after spending half a day trying to compile the generated stubs - not even talking to the Python C library, mind you - I decided to go back to Google.

I thought that, perhaps, someone had written a way to generate JNI headers and native methods. There are a few options out there, actually, but the one I settled on is JNA. It's open source, cross-platform[1], and surprisingly easy to use. An added benefit is that Timothy Wall, a major project contributor, is pretty responsive on the user mailing list.

It's simple to use:


package com.sun.jna.examples;

import com.sun.jna.Library;
import com.sun.jna.Native;
import com.sun.jna.Platform;

/** Simple example of native library declaration and usage. */
public class HelloWorld {

public interface CLibrary extends Library {
CLibrary INSTANCE = (CLibrary)
Native.loadLibrary(
(Platform.isWindows() ? "msvcrt" : "c"),
CLibrary.class);

void printf(String format, Object... args);
}

public static void main(String[] args) {
CLibrary.INSTANCE.printf("Hello, World\n");
for (int i=0;i < args.length;i++) {
CLibrary.INSTANCE.printf(
"Argument %d: %s\n", i, args[i]);
}
}
}


That's it. You now have native access to printf. All of the native calls are handled through a proxy generated at run time. You don't have to write all the methods in the library, either. The resolution is done on the fly. If you need to add another native call, simply add it to the interface.

The library begins to get a little hairy when dealing with pointers and references. For the most part, pointers are handled for you (it's Java, every things a pointer). But what about primitives, you ask? JNA solves this through a nice set of Pointer and PointerType classes.

So now if you have a C function like the following:


void setSomeValue(int* retValue)


you can map it via the following:

public interface MyNativeLibrary extends Library {
...

public void setSomeValue(IntByRefrence retValue);

...
}


In Java, we can now call the method like so:

IntByReference retValue = new IntByReference();
MyNativeLibrary.INSTANCE.setSomeValue(retValue);
System.out.println("retValue = " + retValue.getValue());


There is also a PointerByReference, which can be used to do the same thing with pointer-based arrays, or String values.

Structures and callbacks are also very simple to map. I'll take a look at these in a followup post.

I don't know why anyone would write JNI anymore[2]. This library has made access to native code amazingly simple, and surprisingly intuitive. I would would highly recommend it to anyone who is avoiding a solution, simply because they don't want to make native calls.


[1] Linux, Mac OS X, and Windows, for now.

[2] Ok, there is some overhead, but through clever caching of the function pointers, it's pretty fast. I'll have to find some numbers on it.