com.bigdata.service
Class ResourceService

java.lang.Object
  com.bigdata.service.ResourceService

Direct Known Subclasses:

ManagedResourceService

public abstract class ResourceService
extends Object

A service which permits resources (managed files or buffers) identified by a UUID to be read by a remote service. This class runs one thread to accept connections and thread pool to send data.

Version:

$Id: ResourceService.java 3654 2010-09-28 11:50:52Z thompsonbry $

Author:

Bryan Thompson

TODO:

This class was derived from the ResourceService in the HA branch. The NIO pieces of this class should be based on the work in the HA branch on the write replication pipeline. The main structure of this class is based on the ResourceService since it allows concurrent connections, which the write pipeline does not support (because it is designed specifically for a node to node streaming replication protocol), however the class as originally implemented does not allow us to keep connections open., Nodes should hold open connections for buffer transfers to avoid overhead with establishing those connections and problems with immediate reconnect under heavy load., Should have start() method to defer initialization of thread pools and the ServerSocket until outside of the constructor. Either handle visibility appropriately for those fields or move them and the shutdown protocol into an inner class which is only initialized when the service is running., Verify that multiple transfers may proceed in parallel to the from the same source to the same receiver. This is important in order for low latency queries to remain lively when we are moving a shard from one node to another. This also suggests that we may want to have different pools for servicing buffer requests for low latency queries versus requests for high latency queries and shards., If we had an Adler32 native implementation then we could checksum the nio ByteBuffers without copying the data into the Java heap. The Adler32 implementation is based on zlib, which is an acceptable license. So we might create a JNI component for that. Also, there may be direct JNI calls available into the underlying zlib library from Java.

This could be created in a fairly straightforward manner by bundling zlib, bundling a Make file for our JNI components, and using GetDirectByteBufferAddress to access the data on the C heap. See JNI Spec, JNI FAQ, and ZLib project., The change log for this class is as follows:

- hold open connections in order to avoid problems with socket reconnects;

- exchange buffers using NIO transfers rather than sending files using standard file IO and streams.

- done. make the counter updates thread safe using CATs.

- added generics to the base class used to fetch files or buffers.

- added a ResourceService.ResourceTypeEnum byte in the request protocol (breaks compatibility for earlier versions of this service).

Nested Class Summary

static class	ResourceService.Counters Performance counters for the ResourceService.
static class	ResourceService.FetchResourceTask<S,T> Client for a BufferService reads a single resource from the specified service, writing it into the local file system.
static class	ResourceService.ReadBufferTask Class sends a request for a remote ByteBuffer and then receives the data into a local ByteBuffer.
static class	ResourceService.ReadResourceTask Task sends a request for a file's data and then receives the data onto a local file.
static class	ResourceService.ResourceTypeEnum Type safe enumeration of the kinds of resources which can be served.
static class	ResourceService.StatusEnum Known status codes.

Field Summary

ResourceService.Counters	counters Performance counters for this service.
protected static org.apache.log4j.Logger	log

Constructor Summary

ResourceService(InetSocketAddress addr, int requestServicePoolSize)
Create and start the service.

Method Summary

void	awaitRunning(long timeout, TimeUnit unit) Wait until the service is running.
protected void	finalize() Overridden to ensure that the service is always shutdown.
InetSocketAddress	getAddr() Return the address at which this service is accepting connections.
protected abstract ByteBuffer	getBuffer(UUID uuid) Return ByteBuffer identified by the UUID.
protected abstract File	getResource(UUID uuid) Return file identified by the UUID.
boolean	isOpen()
void	shutdown()
void	shutdownNow()

Methods inherited from class java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

log

protected static final org.apache.log4j.Logger log

counters

public final ResourceService.Counters counters

Performance counters for this service.

Constructor Detail

ResourceService

public ResourceService(InetSocketAddress addr,
                       int requestServicePoolSize)
                throws IOException

Create and start the service.

Parameters:

addr - The IP address and port at which the service will accept connections. The port MAY be zero to use an ephemeral port.

requestServicePoolSize - The size of the thread pool that will handle requests. When ZERO (0) a cached thread pool will be used with no specific size limit.

Throws:

IOException

Method Detail

getAddr

public InetSocketAddress getAddr()

Return the address at which this service is accepting connections.

finalize

protected void finalize()
                 throws Throwable

Overridden to ensure that the service is always shutdown.

Overrides:

finalize in class Object

Throws:

Throwable

awaitRunning

public void awaitRunning(long timeout,
                         TimeUnit unit)
                  throws InterruptedException,
                         TimeoutException

Wait until the service is running.

Parameters:

timeout - The timeout.

unit - The unit for the timeout.

Throws:

InterruptedException - if interrupted while waiting.

TimeoutException - if the timeout expired before the service was running.

isOpen

public boolean isOpen()

shutdown

public void shutdown()

TODO:

Is there a possible lock ordering problem with synchronized shutdown() and shutdownNow() methods and the internal lock?

shutdownNow

public void shutdownNow()

getResource

protected abstract File getResource(UUID uuid)
                             throws Exception

Return file identified by the UUID.

Parameters:

uuid -

Returns:

The file -or- null if there is no file associated with that UUID.

Throws:

Exception - if the resource may not be served.

getBuffer

protected abstract ByteBuffer getBuffer(UUID uuid)
                                 throws Exception

Return ByteBuffer identified by the UUID.

Parameters:

uuid -

Returns:

The ByteBuffer -or- null if there is no buffer associated with that UUID.

Throws:

Exception - if the resource may not be served.