com.bigdata.service.jini
Class JiniFederation<T>

java.lang.Object
  com.bigdata.service.AbstractFederation<T>
      com.bigdata.service.AbstractScaleOutFederation<T>
          com.bigdata.service.AbstractDistributedFederation<T>
              com.bigdata.service.jini.JiniFederation<T>

All Implemented Interfaces:

IIndexManager, IIndexStore, IBigdataFederation<T>, IFederationDelegate<T>, EventListener, net.jini.discovery.DiscoveryListener, net.jini.lookup.ServiceDiscoveryListener

public class JiniFederation<T>
extends AbstractDistributedFederation<T>
implements net.jini.discovery.DiscoveryListener, net.jini.lookup.ServiceDiscoveryListener

Concrete implementation for Jini.

Version:

$Id$

Author:

Bryan Thompson

Nested Class Summary

Nested classes/interfaces inherited from class com.bigdata.service.AbstractScaleOutFederation

AbstractScaleOutFederation.ForceOverflowTask, AbstractScaleOutFederation.PurgeResourcesTask

Nested classes/interfaces inherited from class com.bigdata.service.AbstractFederation

AbstractFederation.ReportTask, AbstractFederation.StartDeferredTasksTask

Field Summary

protected Condition	discoveryEvent Condition signaled any time there is a DiscoveryEvent delivered to our DiscoveryListener.
protected ReentrantLock	discoveryEventLock Lock controlling access to the discoveryEvent Condition.

Fields inherited from class com.bigdata.service.AbstractScaleOutFederation

metadataIndexCachePolicy

Fields inherited from class com.bigdata.service.AbstractFederation

log

Constructor Summary

JiniFederation(JiniClient<T> client, JiniClientConfig jiniConfig, ZookeeperClientConfig zooConfig)
Initiates discovery for one or more service registrars and establishes a lookup caches for various bigdata services.

Method Summary

boolean	awaitJiniRegistrars(long timeout, TimeUnit unit) Await at least one jini ServiceRegistrar to be discovered.
void	cancelMonitoredTasks(boolean mayInterruptIfRunning) Cancel any monitored tasks which are still running.
void	createKeyZNodes(org.apache.zookeeper.ZooKeeper zookeeper) Create key znodes used by the federation.
void	destroy() Destroys all discovered services belonging to the federation and their persistent data and disconnects from the federation.
void	discarded(net.jini.discovery.DiscoveryEvent e) Signals anyone waiting on discoveryEvent.
void	discovered(net.jini.discovery.DiscoveryEvent e) Signals anyone waiting on discoveryEvent.
void	distributedFederationShutdown(boolean immediateShutdown) Shutdown the services in the distributed federation NOT just your client.
IDataService	getAnyDataService() Return ANY IDataService which has been (or could be) discovered and which is part of the connected federation.
JiniClient<T>	getClient() Return the client object that was used to connect to the federation.
ClientServicesClient	getClientServicesClient() Cached lookup for discovered IClientServices.
IDataService	getDataService(UUID serviceUUID) Resolve the service identifier to an IDataService.
IDataService	getDataServiceByName(String name) Return an IDataService joined with this IBigdataFederation and having the specified service name.
DataServicesClient	getDataServicesClient() Cached lookup for discovered IDataServices and IMetadataServices.
UUID[]	getDataServiceUUIDs(int maxCount) Return an array UUIDs for IDataServices.
net.jini.discovery.LookupDiscoveryManager	getDiscoveryManagement() An object used to manage jini service registrar discovery.
protected net.jini.export.Exporter	getExporter(boolean enableDGC) Return an Exporter for a single object that implements one or more Remote interfaces.
long	getLastCommitTime() Return the last commit time for the federation (the timestamp of the most recent commit point across all IDataServices).
ILoadBalancerService	getLoadBalancerService() Return the load balancer service (or a proxy for that service).
IMetadataService	getMetadataService() Return the metadata service (or a proxy for the metadata service).
<E> E	getProxy(E obj, boolean enableDGC) Return a proxy for an object.
<E> Future<E>	getProxy(Future<E> future) Note that Futures generated by java.util.concurrent are NOT Serializable.
<E> IAsynchronousIterator<E>	getProxy(IAsynchronousIterator<E> sourceIterator, IStreamSerializer<E> serializer, int capacity) Export and return a proxy object.
<E> IBuffer<E>	getProxy(IBuffer<E> buffer) A proxy for an IBuffer that does not extend Remote and which DOES NOT declare that its methods throw IOException (for RMI).
<E,V> IRunnableBuffer<E>	getProxy(IRunnableBuffer<E> buffer) A proxy for an IRunnableBuffer that does not extend Remote and which DOES NOT declare that its methods throw IOException (for RMI).
IResourceLockService	getResourceLockService() The service that may be used to acquire synchronous distributed locks without deadlock detection.
net.jini.lookup.ServiceDiscoveryManager	getServiceDiscoveryManager() An object used to lookup services using the discovered service registars.
ServicesManagerClient	getServicesManagerClient()
ITransactionService	getTransactionService() Return the ITransactionService (or a proxy for that service).
ZookeeperClientConfig	getZooConfig() Return the zookeeper client configuration.
org.apache.zookeeper.ZooKeeper	getZookeeper() Return a ZooKeeper client.
ZooKeeperAccessor	getZookeeperAccessor() Return an object that may be used to obtain a ZooKeeper client and that may be used to obtain the a new ZooKeeper client if the current session has been expired (an absorbing state for the ZooKeeper client).
void	serviceAdded(net.jini.lookup.ServiceDiscoveryEvent e) Invokes #serviceJoin(com.bigdata.service.IService, UUID, String)
void	serviceChanged(net.jini.lookup.ServiceDiscoveryEvent e) NOP.
void	serviceRemoved(net.jini.lookup.ServiceDiscoveryEvent e) Invokes AbstractFederation.serviceLeave(UUID)
void	shutdown() Normal shutdown allows any existing client requests to federation services to complete but does not schedule new requests, disconnects from the federation, and then terminates any background processing that is being performed on the behalf of the client (service discovery, etc).
void	shutdownNow() Immediate shutdown terminates any client requests to federation services, disconnects from the federation, and then terminate any background processing that is being performed on the behalf of the client (service discovery, etc).
<T1> Future<T1>	submitMonitoredTask(Callable<T1> task) Submits the task for execution and monitors its Future.

Methods inherited from class com.bigdata.service.AbstractDistributedFederation

isDistributed, isStable

Methods inherited from class com.bigdata.service.AbstractScaleOutFederation

awaitServices, forceOverflow, getIndex, getIndexCache, getMetadataIndex, getMetadataIndexCache, isScaleOut, locatorScan

Methods inherited from class com.bigdata.service.AbstractFederation

addScheduledTask, assertOpen, didStart, dropIndex, getCounterSet, getDataServices, getExecutorService, getGlobalFileSystem, getGlobalRowStore, getHostCounterSet, getHttpdURL, getIndexCounters, getResourceLocator, getScheduledExecutorService, getService, getServiceCounterPathPrefix, getServiceCounterPathPrefix, getServiceCounterSet, getServiceIface, getServiceName, getServiceUUID, getTaskCounters, getTempStore, isOpen, isServiceReady, newHttpd, reattachDynamicCounters, registerIndex, registerIndex, registerIndex, reportCounters, sendEvent, serviceJoin, serviceLeave

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

discoveryEventLock

protected final ReentrantLock discoveryEventLock

Lock controlling access to the discoveryEvent Condition.

discoveryEvent

protected final Condition discoveryEvent

Condition signaled any time there is a DiscoveryEvent delivered to our DiscoveryListener.

Constructor Detail

JiniFederation

public JiniFederation(JiniClient<T> client,
                      JiniClientConfig jiniConfig,
                      ZookeeperClientConfig zooConfig)

Initiates discovery for one or more service registrars and establishes a lookup caches for various bigdata services.

Parameters:

client - The client.

Method Detail

getZooConfig

public ZookeeperClientConfig getZooConfig()

Return the zookeeper client configuration.

getZookeeperAccessor

public ZooKeeperAccessor getZookeeperAccessor()

Return an object that may be used to obtain a ZooKeeper client and that may be used to obtain the a new ZooKeeper client if the current session has been expired (an absorbing state for the ZooKeeper client).

getZookeeper

public org.apache.zookeeper.ZooKeeper getZookeeper()

Return a ZooKeeper client.

Note: This is a shorthand for obtaining a valid ZooKeeper client from the ZooKeeperAccessor. If the session associated with the current ZooKeeper client is expired, then a distinct ZooKeeper client associated with a distinct session will be returned. See getZookeeperAccessor() which lets you explicitly handle a KeeperException.SessionExpiredException or the ZooKeeper ZooKeeper.States#CLOSED state.

See Also:

getZookeeperAccessor()

TODO:

timeout variant w/ unit?

createKeyZNodes

public void createKeyZNodes(org.apache.zookeeper.ZooKeeper zookeeper)
                     throws org.apache.zookeeper.KeeperException,
                            InterruptedException

Create key znodes used by the federation.

Throws:

org.apache.zookeeper.KeeperException

InterruptedException

TODO:

probably better written using a timeout than the caller's zk inst.

getDiscoveryManagement

public net.jini.discovery.LookupDiscoveryManager getDiscoveryManagement()

An object used to manage jini service registrar discovery.

getServiceDiscoveryManager

public net.jini.lookup.ServiceDiscoveryManager getServiceDiscoveryManager()

An object used to lookup services using the discovered service registars.

getDataServicesClient

public DataServicesClient getDataServicesClient()

Cached lookup for discovered IDataServices and IMetadataServices. Will block on a cache miss and attempt to discover an appropriate service instance.

getServicesManagerClient

public ServicesManagerClient getServicesManagerClient()

getClientServicesClient

public ClientServicesClient getClientServicesClient()

Cached lookup for discovered IClientServices.

awaitJiniRegistrars

public boolean awaitJiniRegistrars(long timeout,
                                   TimeUnit unit)
                            throws InterruptedException

Await at least one jini ServiceRegistrar to be discovered.

Parameters:

timeout -

unit -

Returns:

true at least one registrar was available before the timeout.

Throws:

InterruptedException - if interrupted awaiting a jini ServiceRegistrar to be discovered.

getClient

public JiniClient<T> getClient()

Description copied from interface: IBigdataFederation

Return the client object that was used to connect to the federation.

Specified by:

getClient in interface IBigdataFederation<T>

Overrides:

getClient in class AbstractFederation<T>

getLoadBalancerService

public ILoadBalancerService getLoadBalancerService()

Description copied from interface: IBigdataFederation

Return the load balancer service (or a proxy for that service).

Specified by:

getLoadBalancerService in interface IBigdataFederation<T>

Returns:

The service -or- null if the service has not been discovered.

getTransactionService

public ITransactionService getTransactionService()

Description copied from interface: IBigdataFederation

Return the ITransactionService (or a proxy for that service).

Specified by:

getTransactionService in interface IBigdataFederation<T>

Returns:

The service -or- null if the service has not been discovered.

getResourceLockService

public IResourceLockService getResourceLockService()

Description copied from interface: IIndexStore

The service that may be used to acquire synchronous distributed locks without deadlock detection.

Specified by:

getResourceLockService in interface IIndexStore

getMetadataService

public IMetadataService getMetadataService()

Description copied from interface: IBigdataFederation

Return the metadata service (or a proxy for the metadata service).

Specified by:

getMetadataService in interface IBigdataFederation<T>

Returns:

The service -or- null if the service has not been discovered.

getDataServiceUUIDs

public UUID[] getDataServiceUUIDs(int maxCount)

Description copied from interface: IBigdataFederation

Return an array UUIDs for IDataServices.

Specified by:

getDataServiceUUIDs in interface IBigdataFederation<T>

Parameters:

maxCount - The maximum #of data services whose UUIDs will be returned. When zero (0) the UUID for all known data services will be returned.

Returns:

An array of UUIDs for data services.

getDataService

public IDataService getDataService(UUID serviceUUID)

Description copied from interface: IBigdataFederation

Resolve the service identifier to an IDataService.

Note: Whether the returned object is a proxy or the service implementation depends on whether the federation is embedded (in process) or distributed (networked).

Specified by:

getDataService in interface IBigdataFederation<T>

Parameters:

serviceUUID - The identifier for a IDataService.

Returns:

The IDataService or null iff the IDataService could not be discovered from its identifier.

getAnyDataService

public IDataService getAnyDataService()

Description copied from interface: IBigdataFederation

Return ANY IDataService which has been (or could be) discovered and which is part of the connected federation.

Note: This method is here as a failsafe when the ILoadBalancerService is not available.

Specified by:

getAnyDataService in interface IBigdataFederation<T>

Returns:

null if there are NO known IDataServices.

getDataServiceByName

public IDataService getDataServiceByName(String name)

Description copied from interface: IBigdataFederation

Return an IDataService joined with this IBigdataFederation and having the specified service name. Services that are not joined will not be discovered.

Note: At least some service fabrics (such as jini) do not enforce a uniqueness constraint on the service name(s). In such cases an arbitrary IDataService method the other requirements will be returned. It is the responsibility of the administrator to ensure that each IDataService is assigned a distinct service name.

Specified by:

getDataServiceByName in interface IBigdataFederation<T>

Parameters:

name - The service name.

Returns:

A service assigned that name -or- null if none is joined with the IBigdataFederation at this time.

shutdown

public void shutdown()

Description copied from class: AbstractFederation

Normal shutdown allows any existing client requests to federation services to complete but does not schedule new requests, disconnects from the federation, and then terminates any background processing that is being performed on the behalf of the client (service discovery, etc).

Note: concrete implementations MUST extend this method.

Note: Clients use IBigdataClient.disconnect(boolean) to disconnect from a federation. The federation implements that disconnect using either AbstractFederation.shutdown() or AbstractFederation.shutdownNow().

The implementation must be a NOP if the federation is already shutdown.

Overrides:

shutdown in class AbstractScaleOutFederation<T>

shutdownNow

public void shutdownNow()

Description copied from class: AbstractFederation

Immediate shutdown terminates any client requests to federation services, disconnects from the federation, and then terminate any background processing that is being performed on the behalf of the client (service discovery, etc).

Note: concrete implementations MUST extend this method to either disconnect from the remote federation or close the embedded federation and then clear the #fed reference so that the client is no longer "connected" to the federation.

Note: Clients use IBigdataClient.disconnect(boolean) to disconnect from a federation. The federation implements that disconnect using either AbstractFederation.shutdown() or AbstractFederation.shutdownNow().

The implementation must be a NOP if the federation is already shutdown.

Overrides:

shutdownNow in class AbstractScaleOutFederation<T>

distributedFederationShutdown

public void distributedFederationShutdown(boolean immediateShutdown)
                                   throws InterruptedException

Shutdown the services in the distributed federation NOT just your client. This method may be used to take the entire federation out of service. All services will be halted and all clients will be disconnected. If you only wish to disconnect from a federation, then use shutdown() or shutdownNow() instead.

The shutdown protocol is as follows:

1. ITransactionService (blocks until shutdown).
2. IDataServices (blocks until all are shutdown).
3. IMetadataService
4. ILoadBalancerService

Parameters:

immediateShutdown - When true the services will be shutdown without waiting for existing transactions and other tasks to terminate.

Throws:

InterruptedException

TODO:

javadoc update.

destroy

public void destroy()

Description copied from interface: IBigdataFederation

Destroys all discovered services belonging to the federation and their persistent data and disconnects from the federation.

Specified by:

destroy in interface IIndexStore

Specified by:

destroy in interface IBigdataFederation<T>

Overrides:

destroy in class AbstractFederation<T>

getLastCommitTime

public long getLastCommitTime()

Description copied from interface: IBigdataFederation

Return the last commit time for the federation (the timestamp of the most recent commit point across all IDataServices).

This is useful for ITx.READ_COMMITTED operations that need to use a consistent timestamp across a series of DataServices or a series of requests against a single DataService that must use a consistent view.

Specified by:

getLastCommitTime in interface IIndexStore

Specified by:

getLastCommitTime in interface IBigdataFederation<T>

Returns:

The timestamp of the most recent commit on the store or 0L iff there have been no commits.

See Also:

IRootBlockView.getLastCommitTime()

getExporter

protected net.jini.export.Exporter getExporter(boolean enableDGC)

Return an Exporter for a single object that implements one or more Remote interfaces.

Note: This uses TCP Server sockets.

Note: This uses [port := 0], which means a random port is assigned.

Note: The VM WILL NOT be kept alive by the exported proxy (keepAlive is false).

Parameters:

enableDGC - if distributed garbage collection should be used for the object to be exported.

Returns:

The Exporter.

getProxy

public <E> IAsynchronousIterator<E> getProxy(IAsynchronousIterator<E> sourceIterator,
                                             IStreamSerializer<E> serializer,
                                             int capacity)

Export and return a proxy object.

Note: This is used for query evaluation in which there is at least one JOIN. It DOES NOT get used for simple IAccessPath scans. Those use IRangeQuery.rangeIterator() instead.

Specified by:

getProxy in class AbstractDistributedFederation<T>

Parameters:

sourceIterator - The source iterator. Note that the iterator normally visits elements of some array type (chunks).

serializer - The object responsible for (de-)serializing a chunk of elements visited by the iterator.

capacity - The capacity for the internal buffer that is used to asynchronously transfer elements (chunks) from the remote iterator to the client iterator.

Returns:

Either a thick iterator (when the results would fit within a single chunk) or a thin iterator that uses RMI to fetch chunks from the remote IAsynchronousIterator.

TODO:

Allow Configuration of the Exporter for the proxy iterators and futures.

Since the Exporter is paired to a single object the configuration of the Exporter will require an additional level of indirection when compared to the Configuration of an AbstractServer's Exporter.

getProxy

public <E> Future<E> getProxy(Future<E> future)

Note that Futures generated by java.util.concurrent are NOT Serializable. Futher note the proxy as generated by an Exporter MUST be encapsulated so that the object returned to the caller can implement Future without having to declare that the methods throw IOException (for RMI).

Specified by:

getProxy in class AbstractDistributedFederation<T>

Parameters:

future - The future.

Returns:

A proxy for that Future that masquerades any RMI exceptions.

getProxy

public <E> IBuffer<E> getProxy(IBuffer<E> buffer)

A proxy for an IBuffer that does not extend Remote and which DOES NOT declare that its methods throw IOException (for RMI).

Specified by:

getProxy in class AbstractDistributedFederation<T>

Parameters:

buffer - The buffer.

Returns:

A proxy for that IBuffer that masquerades any RMI exceptions.

getProxy

public <E,V> IRunnableBuffer<E> getProxy(IRunnableBuffer<E> buffer)

A proxy for an IRunnableBuffer that does not extend Remote and which DOES NOT declare that its methods throw IOException (for RMI).

Parameters:

buffer - The buffer.

Returns:

A proxy for that IBuffer that masquerades any RMI exceptions.

getProxy

public <E> E getProxy(E obj,
                      boolean enableDGC)

Description copied from class: AbstractDistributedFederation

Return a proxy for an object.

Specified by:

getProxy in class AbstractDistributedFederation<T>

Parameters:

obj - The object.

enableDGC - If distributed garbage collection should be used for the object.

Returns:

The proxy.

serviceAdded

public void serviceAdded(net.jini.lookup.ServiceDiscoveryEvent e)

Invokes #serviceJoin(com.bigdata.service.IService, UUID, String)

Specified by:

serviceAdded in interface net.jini.lookup.ServiceDiscoveryListener

serviceChanged

public void serviceChanged(net.jini.lookup.ServiceDiscoveryEvent e)

NOP.

Specified by:

serviceChanged in interface net.jini.lookup.ServiceDiscoveryListener

serviceRemoved

public void serviceRemoved(net.jini.lookup.ServiceDiscoveryEvent e)

Invokes AbstractFederation.serviceLeave(UUID)

Specified by:

serviceRemoved in interface net.jini.lookup.ServiceDiscoveryListener

discarded

public void discarded(net.jini.discovery.DiscoveryEvent e)

Signals anyone waiting on discoveryEvent.

Specified by:

discarded in interface net.jini.discovery.DiscoveryListener

discovered

public void discovered(net.jini.discovery.DiscoveryEvent e)

Signals anyone waiting on discoveryEvent.

Specified by:

discovered in interface net.jini.discovery.DiscoveryListener

submitMonitoredTask

public <T1> Future<T1> submitMonitoredTask(Callable<T1> task)

Submits the task for execution and monitors its Future.

Parameters:

task - The task.

Returns:

The Future.

TODO:

move this up to AbstractFederation?

cancelMonitoredTasks

public void cancelMonitoredTasks(boolean mayInterruptIfRunning)

Cancel any monitored tasks which are still running.

Parameters:

mayInterruptIfRunning -