com.bigdata.service
Class AbstractFederation<T>

java.lang.Object
  com.bigdata.service.AbstractFederation<T>

Type Parameters:

T - The generic type of the client or service.

All Implemented Interfaces:

IIndexManager, IIndexStore, IBigdataFederation<T>, IFederationDelegate<T>

Direct Known Subclasses:

AbstractScaleOutFederation, LocalDataServiceFederation

public abstract class AbstractFederation<T>
extends Object
implements IBigdataFederation<T>

Abstract base class for IBigdataFederation implementations.

Version:

$Id: AbstractFederation.java 2265 2009-10-26 12:51:06Z thompsonbry $

Author:

Bryan Thompson

TODO:

implement IServiceShutdown. When it is declared here it messes up the Options interface hierarchy. What appears to be happening is that the IServiceShutdown.Options interface is flattened into IServiceShutdown and it shadows the Options that are being used.

Nested Class Summary

static class	AbstractFederation.ReportTask Periodically report performance counter data to the ILoadBalancerService.
protected class	AbstractFederation.StartDeferredTasksTask This task runs periodically.

Field Summary

protected static org.apache.log4j.Logger

log

Constructor Summary

protected

AbstractFederation(IBigdataClient<T> client)

Method Summary

ScheduledFuture	addScheduledTask(Runnable task, long initialDelay, long delay, TimeUnit unit) Adds a task which will run until canceled, until it throws an exception, or until the federation is shutdown().
protected void	assertOpen()
void	destroy() Destroys all discovered services belonging to the federation and their persistent data and disconnects from the federation.
void	didStart() Delegated.
void	dropIndex(String name) Drop a scale-out index.
AbstractClient<T>	getClient() Return the client object that was used to connect to the federation.
CounterSet	getCounterSet() The CounterSet which the client will use report its statistics to the ILoadBalancerService.
IDataService[]	getDataServices(UUID[] uuids) Return an array of IDataService references that is correlated with the given array of IDataService UUIDs.
ExecutorService	getExecutorService() A thread pool that may be used by clients to parallelize operations against the federation.
BigdataFileSystem	getGlobalFileSystem() Return the global file system used to store block-structured files and their metadata and as a source and sink for map/reduce processing.
SparseRowStore	getGlobalRowStore() Return the global SparseRowStore used to store named property sets in the federation.
CounterSet	getHostCounterSet() The node in IBigdataFederation.getCounterSet() corresponding to the root of the host on which the client or service is executing.
String	getHttpdURL() The URL that may be used to access the local httpd service for this client or service.
IClientIndex	getIndex(String name, long timestamp) Applies an AbstractIndexCache and strengthens the return type.
protected abstract AbstractIndexCache<? extends IClientIndex>	getIndexCache() Return the cache for IIndex objects.
ScaleOutIndexCounters	getIndexCounters(String name) Return the ScaleOutIndexCounters for the specified scale-out index for this client.
DefaultResourceLocator	getResourceLocator() Return the default locator for resources that are logical index containers (relations and relation containers).
ScheduledExecutorService	getScheduledExecutorService() A service which may be used to schedule performance counter sampling tasks.
T	getService() Delegated.
String	getServiceCounterPathPrefix() The path prefix under which all of the client or service's counters are located.
static String	getServiceCounterPathPrefix(UUID serviceUUID, Class serviceIface, String hostname) The path prefix under which all of the client or service's counters are located.
CounterSet	getServiceCounterSet() The node in IBigdataFederation.getCounterSet() corresponding to the root of the client or service connected using this federation.
Class	getServiceIface() Delegated.
String	getServiceName() Delegated.
UUID	getServiceUUID() Delegated.
TaskCounters	getTaskCounters() Return the TaskCounters which aggregate across all operations performed by the client against the connected federation.
TemporaryStore	getTempStore() A factory for TemporaryStores.
boolean	isOpen()
boolean	isServiceReady() Delegated.
AbstractHTTPD	newHttpd(int httpdPort, CounterSet counterSet) Delegated.
void	reattachDynamicCounters() Delegated.
void	registerIndex(IndexMetadata metadata) Register a scale-out index.
UUID	registerIndex(IndexMetadata metadata, byte[][] separatorKeys, UUID[] dataServiceUUIDs) Register and statically partition a scale-out index.
UUID	registerIndex(IndexMetadata metadata, UUID dataServiceUUID) Register a scale-out index and assign the initial index partition to the specified data service.
void	reportCounters() Forces the immediate reporting of the CounterSet to the ILoadBalancerService.
protected void	sendEvent(Event e) Queues up an event to be sent to the ILoadBalancerService.
void	serviceJoin(IService service, UUID serviceUUID) Notice that the service has been discovered.
void	serviceLeave(UUID serviceUUID) Notice that the service is no longer available.
void	shutdown() Normal shutdown allows any existing client requests to federation services to complete but does not schedule new requests, disconnects from the federation, and then terminates any background processing that is being performed on the behalf of the client (service discovery, etc).
void	shutdownNow() Immediate shutdown terminates any client requests to federation services, disconnects from the federation, and then terminate any background processing that is being performed on the behalf of the client (service discovery, etc).

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.bigdata.service.IBigdataFederation

getAnyDataService, getDataService, getDataServiceByName, getDataServiceUUIDs, getLastCommitTime, getLoadBalancerService, getMetadataIndex, getMetadataService, getTransactionService, isDistributed, isScaleOut, isStable

Methods inherited from interface com.bigdata.journal.IIndexStore

getResourceLockService

Field Detail

log

protected static final org.apache.log4j.Logger log

Constructor Detail

AbstractFederation

protected AbstractFederation(IBigdataClient<T> client)

Method Detail

getClient

public AbstractClient<T> getClient()

Description copied from interface: IBigdataFederation

Return the client object that was used to connect to the federation.

Specified by:

getClient in interface IBigdataFederation<T>

isOpen

public final boolean isOpen()

shutdown

public void shutdown()

Normal shutdown allows any existing client requests to federation services to complete but does not schedule new requests, disconnects from the federation, and then terminates any background processing that is being performed on the behalf of the client (service discovery, etc).

Note: concrete implementations MUST extend this method.

Note: Clients use IBigdataClient.disconnect(boolean) to disconnect from a federation. The federation implements that disconnect using either shutdown() or shutdownNow().

The implementation must be a NOP if the federation is already shutdown.

shutdownNow

public void shutdownNow()

Immediate shutdown terminates any client requests to federation services, disconnects from the federation, and then terminate any background processing that is being performed on the behalf of the client (service discovery, etc).

Note: concrete implementations MUST extend this method to either disconnect from the remote federation or close the embedded federation and then clear the #fed reference so that the client is no longer "connected" to the federation.

Note: Clients use IBigdataClient.disconnect(boolean) to disconnect from a federation. The federation implements that disconnect using either shutdown() or shutdownNow().

The implementation must be a NOP if the federation is already shutdown.

destroy

public void destroy()

Description copied from interface: IBigdataFederation

Destroys all discovered services belonging to the federation and their persistent data and disconnects from the federation.

Specified by:

destroy in interface IIndexStore

Specified by:

destroy in interface IBigdataFederation<T>

assertOpen

protected void assertOpen()

Throws:

IllegalStateException - if the client has disconnected from the federation.

getScheduledExecutorService

public ScheduledExecutorService getScheduledExecutorService()

A service which may be used to schedule performance counter sampling tasks.

getHttpdURL

public final String getHttpdURL()

Description copied from interface: IBigdataFederation

The URL that may be used to access the local httpd service for this client or service.

Specified by:

getHttpdURL in interface IBigdataFederation<T>

Returns:

The URL -or- null if the httpd service is not running.

getResourceLocator

public DefaultResourceLocator getResourceLocator()

Description copied from interface: IIndexStore

Return the default locator for resources that are logical index containers (relations and relation containers).

Specified by:

getResourceLocator in interface IIndexStore

getTaskCounters

public TaskCounters getTaskCounters()

Return the TaskCounters which aggregate across all operations performed by the client against the connected federation. These TaskCounters are sampled by a ThreadPoolExecutorStatisticsTask and the sampled data are reported by the client to the ILoadBalancerService.

getIndexCounters

public ScaleOutIndexCounters getIndexCounters(String name)

Return the ScaleOutIndexCounters for the specified scale-out index for this client. There is only a single instance per scale-out index and all operations by this client on that index are aggregated by that instance. These counters are reported by the client to the ILoadBalancerService.

Parameters:

name - The scale-out index name.

addScheduledTask

public ScheduledFuture addScheduledTask(Runnable task,
                                        long initialDelay,
                                        long delay,
                                        TimeUnit unit)

Adds a task which will run until canceled, until it throws an exception, or until the federation is shutdown().

Note: Tasks run on this service generally update sampled values on ICounters reported to the ILoadBalancerService. Basic information on the getExecutorService() is reported automatically. Clients may add additional tasks to report on client-side aspects of their application.

Note: Non-sampled counters are automatically conveyed to the ILoadBalancerService once added to the basic CounterSet returned by getCounterSet().

Parameters:

task - The task.

initialDelay - The initial delay.

delay - The delay between invocations.

unit - The units for the delay parameters.

Returns:

The ScheduledFuture for that task.

getCounterSet

public CounterSet getCounterSet()

Description copied from interface: IBigdataFederation

The CounterSet which the client will use report its statistics to the ILoadBalancerService.

Note: Applications MAY add their own counters (within a suitable namespace) to the returned CounterSet in order to report their own performance data to the ILoadBalancerService.

Specified by:

getCounterSet in interface IBigdataFederation<T>

See Also:

IBigdataFederation.getServiceCounterSet(), IBigdataFederation.getServiceCounterPathPrefix()

getHostCounterSet

public CounterSet getHostCounterSet()

Description copied from interface: IBigdataFederation

The node in IBigdataFederation.getCounterSet() corresponding to the root of the host on which the client or service is executing.

Specified by:

getHostCounterSet in interface IBigdataFederation<T>

getServiceCounterSet

public CounterSet getServiceCounterSet()

Description copied from interface: IBigdataFederation

The node in IBigdataFederation.getCounterSet() corresponding to the root of the client or service connected using this federation.

Specified by:

getServiceCounterSet in interface IBigdataFederation<T>

getServiceCounterPathPrefix

public String getServiceCounterPathPrefix()

Description copied from interface: IBigdataFederation

The path prefix under which all of the client or service's counters are located. The returned path prefix is terminated by an ICounterSet.pathSeparator.

Specified by:

getServiceCounterPathPrefix in interface IBigdataFederation<T>

See Also:

IBigdataFederation.getServiceCounterSet()

getServiceCounterPathPrefix

public static String getServiceCounterPathPrefix(UUID serviceUUID,
                                                 Class serviceIface,
                                                 String hostname)

The path prefix under which all of the client or service's counters are located. The returned path prefix is terminated by an ICounterSet.pathSeparator.

Parameters:

serviceUUID - The service UUID.

serviceIface - The primary interface or class for the service.

hostname - The fully qualified name of the host on which the service is running.

getExecutorService

public ExecutorService getExecutorService()

Description copied from interface: IBigdataFederation

A thread pool that may be used by clients to parallelize operations against the federation. This thread pool is automatically used by the ClientIndexView.

Specified by:

getExecutorService in interface IIndexStore

Specified by:

getExecutorService in interface IBigdataFederation<T>

registerIndex

public void registerIndex(IndexMetadata metadata)

Description copied from interface: IBigdataFederation

Register a scale-out index.

Specified by:

registerIndex in interface IIndexManager

Specified by:

registerIndex in interface IBigdataFederation<T>

Parameters:

metadata - The metadata template used to create component indices for BTrees this scale-out index (this also specifies the name of the scale-out index).

registerIndex

public UUID registerIndex(IndexMetadata metadata,
                          UUID dataServiceUUID)

Description copied from interface: IBigdataFederation

Register a scale-out index and assign the initial index partition to the specified data service.

Specified by:

registerIndex in interface IBigdataFederation<T>

Parameters:

metadata - The metadata template used to create component indices for BTrees this scale-out index (this also specifies the name of the scale-out index).

dataServiceUUID - The data service identifier (optional). When null, a data service will be selected automatically. If IndexMetadata.Options#INITIAL_DATA_SERVICE was specified, then the identified service will be used. Otherwise an underutilized service will be selected using the ILoadBalancerService.

Returns:

The UUID of the registered index.

See Also:

IndexMetadata.Options#INITIAL_DATA_SERVICE

registerIndex

public UUID registerIndex(IndexMetadata metadata,
                          byte[][] separatorKeys,
                          UUID[] dataServiceUUIDs)

Description copied from interface: IBigdataFederation

Register and statically partition a scale-out index.

Specified by:

registerIndex in interface IBigdataFederation<T>

Parameters:

metadata - The metadata template used to create component indices for BTrees this scale-out index (this also specifies the name of the scale-out index).

separatorKeys - The array of separator keys. Each separator key is interpreted as an unsigned byte[]. The first entry MUST be an empty byte[]. The entries MUST be in sorted order.

dataServiceUUIDs - The array of data services onto which each partition defined by a separator key will be mapped (optional). When given, the #of entries in this array MUST agree with the #of entries in the separatorKeys array and all entries must be non-null. When not given, the index partitions will be auto-assigned to the discovered data services.

Returns:

The UUID of the scale-out index.

getIndexCache

protected abstract AbstractIndexCache<? extends IClientIndex> getIndexCache()

Return the cache for IIndex objects.

getIndex

public IClientIndex getIndex(String name,
                             long timestamp)

Applies an AbstractIndexCache and strengthens the return type. Obtain a view on a partitioned index.

Specified by:

getIndex in interface IIndexStore

Specified by:

getIndex in interface IBigdataFederation<T>

Parameters:

name - The index name.

timestamp - A transaction identifier, ITx.UNISOLATED for the unisolated index view, ITx.READ_COMMITTED, or timestamp for a historical view no later than the specified timestamp.

Returns:

The index or null if the index does not exist.

dropIndex

public void dropIndex(String name)

Description copied from interface: IBigdataFederation

Drop a scale-out index.

Specified by:

dropIndex in interface IIndexManager

Specified by:

dropIndex in interface IBigdataFederation<T>

Parameters:

name - The index name.

getGlobalRowStore

public SparseRowStore getGlobalRowStore()

Description copied from interface: IBigdataFederation

Return the global SparseRowStore used to store named property sets in the federation.

Specified by:

getGlobalRowStore in interface IIndexStore

Specified by:

getGlobalRowStore in interface IBigdataFederation<T>

See Also:

GlobalRowStoreSchema

getGlobalFileSystem

public BigdataFileSystem getGlobalFileSystem()

Description copied from interface: IIndexStore

Return the global file system used to store block-structured files and their metadata and as a source and sink for map/reduce processing.

Specified by:

getGlobalFileSystem in interface IIndexStore

See Also:

BigdataFileSystem

getTempStore

public TemporaryStore getTempStore()

Description copied from interface: IIndexStore

A factory for TemporaryStores. TemporaryStores are thread-safe and may be used by multiple processes at once. Old TemporaryStores are eventually retired by the factory and their storage is reclaimed once they are finalized (after they are no longer in use by any process). The decision to retire a TemporaryStore is either made implicitly, when it is no longer weakly reachable, or explicitly, when it has grown large enough that no new processes should begin using that TemporaryStore. In the latter case, the TemporaryStore will remain available to the process(es) using it and a new TemporaryStore will be allocated and made available to the caller.

It is important that processes do not hold a hard reference to a TemporaryStore beyond the end of the process as that will prevent the TemporaryStore from being finalized. Holding reference to an AbstractBTree created on a TemporaryStore is equivalent to holding a hard reference to the TemporaryStore itself since the AbstractBTree holds onto the backing IRawStore using a hard reference.

Specified by:

getTempStore in interface IIndexStore

Returns:

A TemporaryStore.

reportCounters

public void reportCounters()

Forces the immediate reporting of the CounterSet to the ILoadBalancerService. Any errors will be logged, not thrown.

getService

public T getService()

Delegated.

Specified by:

getService in interface IFederationDelegate<T>

getServiceName

public String getServiceName()

Delegated.

Specified by:

getServiceName in interface IFederationDelegate<T>

Returns:

A name for the service.

getServiceIface

public Class getServiceIface()

Delegated.

Specified by:

getServiceIface in interface IFederationDelegate<T>

Returns:

The class or interface and never null.

getServiceUUID

public UUID getServiceUUID()

Delegated.

Specified by:

getServiceUUID in interface IFederationDelegate<T>

See Also:

AbstractService.setServiceUUID(UUID)

isServiceReady

public boolean isServiceReady()

Delegated.

Specified by:

isServiceReady in interface IFederationDelegate<T>

reattachDynamicCounters

public void reattachDynamicCounters()

Delegated.

Specified by:

reattachDynamicCounters in interface IFederationDelegate<T>

didStart

public void didStart()

Delegated.

Specified by:

didStart in interface IFederationDelegate<T>

newHttpd

public AbstractHTTPD newHttpd(int httpdPort,
                              CounterSet counterSet)
                       throws IOException

Delegated.

Specified by:

newHttpd in interface IFederationDelegate<T>

counterSet - The root CounterSet that will be served up.

Returns:

The httpd daemon.

Throws:

IOException

serviceJoin

public void serviceJoin(IService service,
                        UUID serviceUUID)

Description copied from interface: IFederationDelegate

Notice that the service has been discovered. This notice will be generated the first time the service is discovered by a given IBigdataClient.

Specified by:

serviceJoin in interface IFederationDelegate<T>

Parameters:

service - The service.

serviceUUID - The service UUID.

serviceLeave

public void serviceLeave(UUID serviceUUID)

Description copied from interface: IFederationDelegate

Notice that the service is no longer available. This notice will be generated once for a given IBigdataClient when the service is no longer available from any of its service registrars.

Specified by:

serviceLeave in interface IFederationDelegate<T>

Parameters:

serviceUUID - The service UUID.

getDataServices

public IDataService[] getDataServices(UUID[] uuids)

Description copied from interface: IBigdataFederation

Return an array of IDataService references that is correlated with the given array of IDataService UUIDs.

Note: This method will also resolve the UUID of an IMetadataService.

Specified by:

getDataServices in interface IBigdataFederation<T>

Returns:

The (meta)data service proxies.

TODO:

it may be possible to optimize this for the jini case.

sendEvent

protected void sendEvent(Event e)

Queues up an event to be sent to the ILoadBalancerService. Events are maintained on a non-blocking queue (no fixed capacity) and sent by a scheduled task.

Parameters:

e -

See Also:

SendEventsTask