How to use LOD with your data

Programmer's documentation > Developing with the JViews Charts SDK > Using Load-On-Demand > How to use LOD with your data

The data connection is performed by means of the IlvDataTileLoader interface.

The complete source file can be found in <installdir>/jviews-charts/samples/lod/src/lod/DataTileLoader.java.

The tile loader that you are going to consider must be in accordance with the following specifications:

Data is stored in a binary file. This file contains 32-bit integer records that represent the y-values of the data points. The x-value of a data point is equal to its index in the file.

The loading of the tiles must be performed in a separate thread.

The loading can be interrupted.

First you need to study the data members of the loader class.

public class DataTileLoader implements IlvDataTileLoader

{ private IlvDataInterval xRange = new IlvDataInterval();

private IlvDataInterval yRange = new IlvDataInterval();

private LinkedList tileQueue = new LinkedList(); private

File file; private LoadThread loadThread;

private long lag;

The xRange and yRange members are used to store the limits of the values provided by the loader. These intervals are returned by the DataTileLoader.getXRange and DataTileLoader.getYRange methods.

The tileQueue list is used as a queue to hold tiles whose loading has been requested.

The file attribute points to the file from which our loader will retrieve the data.

The loadThread member references the thread that the loader will use to load the data.

The lag attribute is used for demonstration purposes to simulate a lag in data loading (the lag will correspond to a duration in milliseconds during which the loading thread will sleep.)

The constructor of our loader has the following implementation:

public DataTileLoader(File file) { parseDataFile(file);

this.file = file; loadThread = new LoadThread(); loadThread.start();

}

The LoadThread.parseDataFile method simply reads the data file contents to initialize the xRange and yRange members. The constructor also creates the loading thread.

private class LoadThread extends Thread {

RandomAccessFile rf = null; IlvDataTile tile; LoadThread()

{ super("TileLoadingThread"); }

The LoadThread class defines two data members:

tile corresponds to the tile being currently processed by the thread.

rf represents the RandomAccessFile instance that is used to retrieve data from the loader data file.

The run method of this thread has the following implementation:

public void run() { while (true) { try

{ rf = null; //==[1]==// tile = popTile();

if (tile == null) continue; //==[2]==// rf =

new RandomAccessFile(DataTileLoader.this.file, "r");

//==[3]==// IlvDataInterval itv = tile.getRange(); long

start = (long) Math.floor(itv.getMin()); if (start < 0) continue;

rf.seek(start*4); int count = (int) itv.getLength()+1;

IlvDataPoints dataPts = new IlvDataPoints(count);

// ... Read the data from the file and fill dataPts. //==[4]==//

tile.setData(dataPts); dataPts.dispose(); tile.loadComplete();

tile = null; //==[5]==// } catch (InterruptedException

e) { // Can be because of cancelLoading. } catch (IOException

e) { // Can be because of cancelLoading. } finally {

// ... Close the file. } }

The run method of the thread consists of an infinite loop whose first instruction is to call the popTile method of the loader ([1]):

private synchronized IlvDataTile popTile()

{ try { while(tileQueue.size() == 0) wait(); // Put loading

thread in waiting state. } catch (InterruptedException x) {

return null; } return (IlvDataTile) tileQueue.removeFirst(); }

Calling this method returns the first tile in the loading queue, or puts the thread into a waiting state if no tile is available. Once the thread has retrieved the tile, a RandomAccessFile object is created ([2]), and the range of the tile is used to read the corresponding data from the file ([3]). Since the x-value of a data point corresponds to the record number, you can easily determine the offset of the first relevant y-value within the file. Note that if x-values were stored in the file, you should have used a more sophisticated algorithm such as a binary search.

Finally, the read data is used to fill the tile, and the loadComplete method is called to notify that the loading is complete ([4]). This whole procedure can be interrupted with the LoadingThread.cancelLoading method, which interrupts the thread and closes the descriptor on the data file. Both cases are handled in the run method so that it actually cancels the loading of the current tile and resumes the loading process, without having to restart the thread ([5]).

Now that you have seen how the loading thread works, you will look at the implementation of the main methods of the IlvDataTileLoader interface, namely the load and release methods.

public synchronized void load(IlvDataTile

tile) throws Exception { pushTile(tile); } private synchronized

void pushTile(IlvDataTile tile) { tileQueue.addLast(tile); if

(tileQueue.size() == 1) notify(); // Wake up loading thread. }

The load method simply appends the tile at the end of the loading queue, and wakes up the loading thread if no tile was previously available for processing (as you have seen in the description of the popTile method the loading thread remains in a waiting state until a loading request is issued).

public synchronized void release(IlvDataTile

tile) { if (loadThread.tile == tile) { // The tile is currently

processed // by the loading thread: cancel. loadThread.cancelLoading();

} else { // Just remove the tile from the queue. tileQueue.remove(tile);

} tile.setData(null); }

The release method cancels the loading request for a given tile, which can be either in the queue or currently being processed by the thread.

In conclusion, it is possible to distinguish two main parts in our tile loader implementation:

The actual data access, which in our case consists of randomly accessing the contents of a file.

The handling of loading requests and the threading technique, which provide an asynchronous and interruptible mechanism.

The DataTileLoader class could be reused with a different data source. For example, data could be retrieved with database queries instead of file access. Also, the implementation of the loading queue could be improved so that it takes into account priorities, instead of following a First-In-First-Out policy.

This implementation highlights two main concerns that should be kept in mind while designing a tile loader:

Since load-on-demand is event-driven, the loading of tiles must be threaded so that it does not hinder the remainder of the application.

In order to be efficient, a tile loader must be able to quickly retrieve the data associated with a given tile. This usually means it is possible either to have random access to data or to issue requests that are processed by smart back-end programs, such as database servers.