com.bigdata.service
Class AbstractFederation<T>

java.lang.Object
  extended by com.bigdata.service.AbstractFederation<T>
Type Parameters:
T - The generic type of the client or service.
All Implemented Interfaces:
IIndexManager, IIndexStore, IBigdataFederation<T>, IFederationDelegate<T>
Direct Known Subclasses:
AbstractScaleOutFederation, LocalDataServiceFederation

public abstract class AbstractFederation<T>
extends Object
implements IBigdataFederation<T>

Abstract base class for IBigdataFederation implementations.

Version:
$Id: AbstractFederation.java 2265 2009-10-26 12:51:06Z thompsonbry $
Author:
Bryan Thompson
TODO:
implement IServiceShutdown. When it is declared here it messes up the Options interface hierarchy. What appears to be happening is that the IServiceShutdown.Options interface is flattened into IServiceShutdown and it shadows the Options that are being used.

Nested Class Summary
static class AbstractFederation.ReportTask
          Periodically report performance counter data to the ILoadBalancerService.
protected  class AbstractFederation.StartDeferredTasksTask
          This task runs periodically.
 
Field Summary
protected static org.apache.log4j.Logger log
           
 
Constructor Summary
protected AbstractFederation(IBigdataClient<T> client)
           
 
Method Summary
 ScheduledFuture addScheduledTask(Runnable task, long initialDelay, long delay, TimeUnit unit)
          Adds a task which will run until canceled, until it throws an exception, or until the federation is shutdown().
protected  void assertOpen()
           
 void destroy()
          Destroys all discovered services belonging to the federation and their persistent data and disconnects from the federation.
 void didStart()
          Delegated.
 void dropIndex(String name)
          Drop a scale-out index.
 AbstractClient<T> getClient()
          Return the client object that was used to connect to the federation.
 CounterSet getCounterSet()
          The CounterSet which the client will use report its statistics to the ILoadBalancerService.
 IDataService[] getDataServices(UUID[] uuids)
          Return an array of IDataService references that is correlated with the given array of IDataService UUIDs.
 ExecutorService getExecutorService()
          A thread pool that may be used by clients to parallelize operations against the federation.
 BigdataFileSystem getGlobalFileSystem()
          Return the global file system used to store block-structured files and their metadata and as a source and sink for map/reduce processing.
 SparseRowStore getGlobalRowStore()
          Return the global SparseRowStore used to store named property sets in the federation.
 CounterSet getHostCounterSet()
          The node in IBigdataFederation.getCounterSet() corresponding to the root of the host on which the client or service is executing.
 String getHttpdURL()
          The URL that may be used to access the local httpd service for this client or service.
 IClientIndex getIndex(String name, long timestamp)
          Applies an AbstractIndexCache and strengthens the return type.
protected abstract  AbstractIndexCache<? extends IClientIndex> getIndexCache()
          Return the cache for IIndex objects.
 ScaleOutIndexCounters getIndexCounters(String name)
          Return the ScaleOutIndexCounters for the specified scale-out index for this client.
 DefaultResourceLocator getResourceLocator()
          Return the default locator for resources that are logical index containers (relations and relation containers).
 ScheduledExecutorService getScheduledExecutorService()
          A service which may be used to schedule performance counter sampling tasks.
 T getService()
          Delegated.
 String getServiceCounterPathPrefix()
          The path prefix under which all of the client or service's counters are located.
static String getServiceCounterPathPrefix(UUID serviceUUID, Class serviceIface, String hostname)
          The path prefix under which all of the client or service's counters are located.
 CounterSet getServiceCounterSet()
          The node in IBigdataFederation.getCounterSet() corresponding to the root of the client or service connected using this federation.
 Class getServiceIface()
          Delegated.
 String getServiceName()
          Delegated.
 UUID getServiceUUID()
          Delegated.
 TaskCounters getTaskCounters()
          Return the TaskCounters which aggregate across all operations performed by the client against the connected federation.
 TemporaryStore getTempStore()
          A factory for TemporaryStores.
 boolean isOpen()
           
 boolean isServiceReady()
          Delegated.
 AbstractHTTPD newHttpd(int httpdPort, CounterSet counterSet)
          Delegated.
 void reattachDynamicCounters()
          Delegated.
 void registerIndex(IndexMetadata metadata)
          Register a scale-out index.
 UUID registerIndex(IndexMetadata metadata, byte[][] separatorKeys, UUID[] dataServiceUUIDs)
          Register and statically partition a scale-out index.
 UUID registerIndex(IndexMetadata metadata, UUID dataServiceUUID)
          Register a scale-out index and assign the initial index partition to the specified data service.
 void reportCounters()
          Forces the immediate reporting of the CounterSet to the ILoadBalancerService.
protected  void sendEvent(Event e)
          Queues up an event to be sent to the ILoadBalancerService.
 void serviceJoin(IService service, UUID serviceUUID)
          Notice that the service has been discovered.
 void serviceLeave(UUID serviceUUID)
          Notice that the service is no longer available.
 void shutdown()
          Normal shutdown allows any existing client requests to federation services to complete but does not schedule new requests, disconnects from the federation, and then terminates any background processing that is being performed on the behalf of the client (service discovery, etc).
 void shutdownNow()
          Immediate shutdown terminates any client requests to federation services, disconnects from the federation, and then terminate any background processing that is being performed on the behalf of the client (service discovery, etc).
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 
Methods inherited from interface com.bigdata.service.IBigdataFederation
getAnyDataService, getDataService, getDataServiceByName, getDataServiceUUIDs, getLastCommitTime, getLoadBalancerService, getMetadataIndex, getMetadataService, getTransactionService, isDistributed, isScaleOut, isStable
 
Methods inherited from interface com.bigdata.journal.IIndexStore
getResourceLockService
 

Field Detail

log

protected static final org.apache.log4j.Logger log
Constructor Detail

AbstractFederation

protected AbstractFederation(IBigdataClient<T> client)
Method Detail

getClient

public AbstractClient<T> getClient()
Description copied from interface: IBigdataFederation
Return the client object that was used to connect to the federation.

Specified by:
getClient in interface IBigdataFederation<T>

isOpen

public final boolean isOpen()

shutdown

public void shutdown()
Normal shutdown allows any existing client requests to federation services to complete but does not schedule new requests, disconnects from the federation, and then terminates any background processing that is being performed on the behalf of the client (service discovery, etc).

Note: concrete implementations MUST extend this method.

Note: Clients use IBigdataClient.disconnect(boolean) to disconnect from a federation. The federation implements that disconnect using either shutdown() or shutdownNow().

The implementation must be a NOP if the federation is already shutdown.


shutdownNow

public void shutdownNow()
Immediate shutdown terminates any client requests to federation services, disconnects from the federation, and then terminate any background processing that is being performed on the behalf of the client (service discovery, etc).

Note: concrete implementations MUST extend this method to either disconnect from the remote federation or close the embedded federation and then clear the #fed reference so that the client is no longer "connected" to the federation.

Note: Clients use IBigdataClient.disconnect(boolean) to disconnect from a federation. The federation implements that disconnect using either shutdown() or shutdownNow().

The implementation must be a NOP if the federation is already shutdown.


destroy

public void destroy()
Description copied from interface: IBigdataFederation
Destroys all discovered services belonging to the federation and their persistent data and disconnects from the federation.

Specified by:
destroy in interface IIndexStore
Specified by:
destroy in interface IBigdataFederation<T>

assertOpen

protected void assertOpen()
Throws:
IllegalStateException - if the client has disconnected from the federation.

getScheduledExecutorService

public ScheduledExecutorService getScheduledExecutorService()
A service which may be used to schedule performance counter sampling tasks.


getHttpdURL

public final String getHttpdURL()
Description copied from interface: IBigdataFederation
The URL that may be used to access the local httpd service for this client or service.

Specified by:
getHttpdURL in interface IBigdataFederation<T>
Returns:
The URL -or- null if the httpd service is not running.

getResourceLocator

public DefaultResourceLocator getResourceLocator()
Description copied from interface: IIndexStore
Return the default locator for resources that are logical index containers (relations and relation containers).

Specified by:
getResourceLocator in interface IIndexStore

getTaskCounters

public TaskCounters getTaskCounters()
Return the TaskCounters which aggregate across all operations performed by the client against the connected federation. These TaskCounters are sampled by a ThreadPoolExecutorStatisticsTask and the sampled data are reported by the client to the ILoadBalancerService.


getIndexCounters

public ScaleOutIndexCounters getIndexCounters(String name)
Return the ScaleOutIndexCounters for the specified scale-out index for this client. There is only a single instance per scale-out index and all operations by this client on that index are aggregated by that instance. These counters are reported by the client to the ILoadBalancerService.

Parameters:
name - The scale-out index name.

addScheduledTask

public ScheduledFuture addScheduledTask(Runnable task,
                                        long initialDelay,
                                        long delay,
                                        TimeUnit unit)
Adds a task which will run until canceled, until it throws an exception, or until the federation is shutdown().

Note: Tasks run on this service generally update sampled values on ICounters reported to the ILoadBalancerService. Basic information on the getExecutorService() is reported automatically. Clients may add additional tasks to report on client-side aspects of their application.

Note: Non-sampled counters are automatically conveyed to the ILoadBalancerService once added to the basic CounterSet returned by getCounterSet().

Parameters:
task - The task.
initialDelay - The initial delay.
delay - The delay between invocations.
unit - The units for the delay parameters.
Returns:
The ScheduledFuture for that task.

getCounterSet

public CounterSet getCounterSet()
Description copied from interface: IBigdataFederation
The CounterSet which the client will use report its statistics to the ILoadBalancerService.

Note: Applications MAY add their own counters (within a suitable namespace) to the returned CounterSet in order to report their own performance data to the ILoadBalancerService.

Specified by:
getCounterSet in interface IBigdataFederation<T>
See Also:
IBigdataFederation.getServiceCounterSet(), IBigdataFederation.getServiceCounterPathPrefix()

getHostCounterSet

public CounterSet getHostCounterSet()
Description copied from interface: IBigdataFederation
The node in IBigdataFederation.getCounterSet() corresponding to the root of the host on which the client or service is executing.

Specified by:
getHostCounterSet in interface IBigdataFederation<T>

getServiceCounterSet

public CounterSet getServiceCounterSet()
Description copied from interface: IBigdataFederation
The node in IBigdataFederation.getCounterSet() corresponding to the root of the client or service connected using this federation.

Specified by:
getServiceCounterSet in interface IBigdataFederation<T>

getServiceCounterPathPrefix

public String getServiceCounterPathPrefix()
Description copied from interface: IBigdataFederation
The path prefix under which all of the client or service's counters are located. The returned path prefix is terminated by an ICounterSet.pathSeparator.

Specified by:
getServiceCounterPathPrefix in interface IBigdataFederation<T>
See Also:
IBigdataFederation.getServiceCounterSet()

getServiceCounterPathPrefix

public static String getServiceCounterPathPrefix(UUID serviceUUID,
                                                 Class serviceIface,
                                                 String hostname)
The path prefix under which all of the client or service's counters are located. The returned path prefix is terminated by an ICounterSet.pathSeparator.

Parameters:
serviceUUID - The service UUID.
serviceIface - The primary interface or class for the service.
hostname - The fully qualified name of the host on which the service is running.

getExecutorService

public ExecutorService getExecutorService()
Description copied from interface: IBigdataFederation
A thread pool that may be used by clients to parallelize operations against the federation. This thread pool is automatically used by the ClientIndexView.

Specified by:
getExecutorService in interface IIndexStore
Specified by:
getExecutorService in interface IBigdataFederation<T>

registerIndex

public void registerIndex(IndexMetadata metadata)
Description copied from interface: IBigdataFederation
Register a scale-out index.

Specified by:
registerIndex in interface IIndexManager
Specified by:
registerIndex in interface IBigdataFederation<T>
Parameters:
metadata - The metadata template used to create component indices for BTrees this scale-out index (this also specifies the name of the scale-out index).

registerIndex

public UUID registerIndex(IndexMetadata metadata,
                          UUID dataServiceUUID)
Description copied from interface: IBigdataFederation
Register a scale-out index and assign the initial index partition to the specified data service.

Specified by:
registerIndex in interface IBigdataFederation<T>
Parameters:
metadata - The metadata template used to create component indices for BTrees this scale-out index (this also specifies the name of the scale-out index).
dataServiceUUID - The data service identifier (optional). When null, a data service will be selected automatically. If IndexMetadata.Options#INITIAL_DATA_SERVICE was specified, then the identified service will be used. Otherwise an underutilized service will be selected using the ILoadBalancerService.
Returns:
The UUID of the registered index.
See Also:
IndexMetadata.Options#INITIAL_DATA_SERVICE

registerIndex

public UUID registerIndex(IndexMetadata metadata,
                          byte[][] separatorKeys,
                          UUID[] dataServiceUUIDs)
Description copied from interface: IBigdataFederation
Register and statically partition a scale-out index.

Specified by:
registerIndex in interface IBigdataFederation<T>
Parameters:
metadata - The metadata template used to create component indices for BTrees this scale-out index (this also specifies the name of the scale-out index).
separatorKeys - The array of separator keys. Each separator key is interpreted as an unsigned byte[]. The first entry MUST be an empty byte[]. The entries MUST be in sorted order.
dataServiceUUIDs - The array of data services onto which each partition defined by a separator key will be mapped (optional). When given, the #of entries in this array MUST agree with the #of entries in the separatorKeys array and all entries must be non-null. When not given, the index partitions will be auto-assigned to the discovered data services.
Returns:
The UUID of the scale-out index.

getIndexCache

protected abstract AbstractIndexCache<? extends IClientIndex> getIndexCache()
Return the cache for IIndex objects.


getIndex

public IClientIndex getIndex(String name,
                             long timestamp)
Applies an AbstractIndexCache and strengthens the return type. Obtain a view on a partitioned index.

Specified by:
getIndex in interface IIndexStore
Specified by:
getIndex in interface IBigdataFederation<T>
Parameters:
name - The index name.
timestamp - A transaction identifier, ITx.UNISOLATED for the unisolated index view, ITx.READ_COMMITTED, or timestamp for a historical view no later than the specified timestamp.
Returns:
The index or null if the index does not exist.

dropIndex

public void dropIndex(String name)
Description copied from interface: IBigdataFederation
Drop a scale-out index.

Specified by:
dropIndex in interface IIndexManager
Specified by:
dropIndex in interface IBigdataFederation<T>
Parameters:
name - The index name.

getGlobalRowStore

public SparseRowStore getGlobalRowStore()
Description copied from interface: IBigdataFederation
Return the global SparseRowStore used to store named property sets in the federation.

Specified by:
getGlobalRowStore in interface IIndexStore
Specified by:
getGlobalRowStore in interface IBigdataFederation<T>
See Also:
GlobalRowStoreSchema

getGlobalFileSystem

public BigdataFileSystem getGlobalFileSystem()
Description copied from interface: IIndexStore
Return the global file system used to store block-structured files and their metadata and as a source and sink for map/reduce processing.

Specified by:
getGlobalFileSystem in interface IIndexStore
See Also:
BigdataFileSystem

getTempStore

public TemporaryStore getTempStore()
Description copied from interface: IIndexStore
A factory for TemporaryStores. TemporaryStores are thread-safe and may be used by multiple processes at once. Old TemporaryStores are eventually retired by the factory and their storage is reclaimed once they are finalized (after they are no longer in use by any process). The decision to retire a TemporaryStore is either made implicitly, when it is no longer weakly reachable, or explicitly, when it has grown large enough that no new processes should begin using that TemporaryStore. In the latter case, the TemporaryStore will remain available to the process(es) using it and a new TemporaryStore will be allocated and made available to the caller.

It is important that processes do not hold a hard reference to a TemporaryStore beyond the end of the process as that will prevent the TemporaryStore from being finalized. Holding reference to an AbstractBTree created on a TemporaryStore is equivalent to holding a hard reference to the TemporaryStore itself since the AbstractBTree holds onto the backing IRawStore using a hard reference.

Specified by:
getTempStore in interface IIndexStore
Returns:
A TemporaryStore.

reportCounters

public void reportCounters()
Forces the immediate reporting of the CounterSet to the ILoadBalancerService. Any errors will be logged, not thrown.


getService

public T getService()
Delegated.

Specified by:
getService in interface IFederationDelegate<T>

getServiceName

public String getServiceName()
Delegated.

Specified by:
getServiceName in interface IFederationDelegate<T>
Returns:
A name for the service.

getServiceIface

public Class getServiceIface()
Delegated.

Specified by:
getServiceIface in interface IFederationDelegate<T>
Returns:
The class or interface and never null.

getServiceUUID

public UUID getServiceUUID()
Delegated.

Specified by:
getServiceUUID in interface IFederationDelegate<T>
See Also:
AbstractService.setServiceUUID(UUID)

isServiceReady

public boolean isServiceReady()
Delegated.

Specified by:
isServiceReady in interface IFederationDelegate<T>

reattachDynamicCounters

public void reattachDynamicCounters()
Delegated.

Specified by:
reattachDynamicCounters in interface IFederationDelegate<T>

didStart

public void didStart()
Delegated.

Specified by:
didStart in interface IFederationDelegate<T>

newHttpd

public AbstractHTTPD newHttpd(int httpdPort,
                              CounterSet counterSet)
                       throws IOException
Delegated.

Specified by:
newHttpd in interface IFederationDelegate<T>
counterSet - The root CounterSet that will be served up.
Returns:
The httpd daemon.
Throws:
IOException

serviceJoin

public void serviceJoin(IService service,
                        UUID serviceUUID)
Description copied from interface: IFederationDelegate
Notice that the service has been discovered. This notice will be generated the first time the service is discovered by a given IBigdataClient.

Specified by:
serviceJoin in interface IFederationDelegate<T>
Parameters:
service - The service.
serviceUUID - The service UUID.

serviceLeave

public void serviceLeave(UUID serviceUUID)
Description copied from interface: IFederationDelegate
Notice that the service is no longer available. This notice will be generated once for a given IBigdataClient when the service is no longer available from any of its service registrars.

Specified by:
serviceLeave in interface IFederationDelegate<T>
Parameters:
serviceUUID - The service UUID.

getDataServices

public IDataService[] getDataServices(UUID[] uuids)
Description copied from interface: IBigdataFederation
Return an array of IDataService references that is correlated with the given array of IDataService UUIDs.

Note: This method will also resolve the UUID of an IMetadataService.

Specified by:
getDataServices in interface IBigdataFederation<T>
Returns:
The (meta)data service proxies.
TODO:
it may be possible to optimize this for the jini case.

sendEvent

protected void sendEvent(Event e)
Queues up an event to be sent to the ILoadBalancerService. Events are maintained on a non-blocking queue (no fixed capacity) and sent by a scheduled task.

Parameters:
e -
See Also:
SendEventsTask


Copyright © 2006-2009 SYSTAP, LLC. All Rights Reserved.